Modular intelligent transportation system

ABSTRACT

A modular intelligent transportation system, comprising an environmentally protected enclosure, a system communications bus, a processor module, communicating with said bus, having a image data input and an audio input, the processor module analyzing the image data and/or audio input for data patterns represented therein, having at least one available option slot, a power supply, and a communication link for external communications, in which at least one available option slot can be occupied by a wireless local area network access point, having a communications path between said communications link and said wireless access point, or other modular components.

CROSS REFERENCE TO RELATED APPLICATION

The present application is a Continuation of U.S. patent applicationSer. No. 13/185,991, filed Jul. 19, 2011, which is a Continuation ofU.S. patent application Ser. No. 11/267,761, filed Nov. 3, 2005, nowU.S. Pat. No. 7,983,835, issued Jul. 19, 2011, which claims benefit ofpriority from U.S. Provisional Patent Application 60/522,748 filed Nov.3, 2004.

BACKGROUND OF THE INVENTION

The invention generally relates to a Modular Intelligent TransportationSystem (MITS), having an open interface for modularity and/orexpandability.

There have been a number of different proposals for intelligenttransportation systems, including roadside systems for monitoringconditions and communicating with nearby vehicles and/or a remotelocation.

These systems are, however, proprietary and generally incompatible, inthat they are not intended to be synergistically or efficiently usedtogether. Intelligent Transportation Systems (ITS) generally must beindustrially and environmentally hardened, to meet the rigors of fielddeployment and have a long anticipated mean time between failures(MTBF). This requires, for example, that the system reside in anenvironmentally sealed enclosure, that the system be capable ofwithstanding extended heat and cold, numerous temperature cycles, and beserviced by a trained, but not necessarily expert technician. Theenclosure itself may be located on the ground, mounted to a utilitypole, building or other structure, suspended in the air by a cable or inany other place where there is access to a power source, includingsolar. In any case, the system may be subject to harsh weatherconditions, lightning strikes, and cable terminations may reveal groundloops or various forms of interference. Thus, electronic systemsintended for ITS are generally dissimilar from consumer or officeelectronics with respect to form factor, electrical interfacespecifications, and more generally, design. Likewise, because of thevertically integrated markets and proprietary designs of ITS systems,there tends not to be standardization of electronics between differentmanufacturers.

One use of ITS modules is for incident monitoring, for example todirectly or inferentially determine whether an accident has occurred.Other applications for ITS modules include communications, Internetaccess, entertainment, accident avoidance, network cruise control, andthe like.

Traffic accidents cause significant costs in terms of direct loss,consequential loss, and societal loss due to obstruction of the roadwayin the aftermath of an accident. Another issue is the allocation ofdirect costs, for example when more than one vehicle is involved, thevehicle at fault is generally held liable for the damages.

It is possible to monitor locations that are likely places for accidentsto occur, however, without intelligence, this process may be inefficientand unproductive. Likewise, without immediate and efficientcommunication of the information obtained, benefits of the monitoringare quite limited.

Since cellular telephone technology has become so widely adopted, themost common means by which motor vehicle accidents are reported toagencies in the U.S. is through cellular telephones. However, this isnot always reliable or immediate if the victims are unable to use theircellular phones or if there are no witnesses with cellular phones toreport the accident, and it fails to record an actual record of theaccident, which can later be used as evidence.

Automobile accident detection systems are common in the art. Upon theoccurrence of an automobile accident, it may be desirable to obtainvideo images and sounds of the accident and to record the time of theaccident and the status of the traffic lights at the time the accidentoccurred. This information can then be sent to a remote location whereemergency crews can be dispatched and the information further examinedand forwarded to authorities in order to determine fault and liability.

A number of prior art techniques are available for predicting theoccurrence of an accident. Some of these require an extended period oftime for an automated system to analyze the data, and thus any reportgenerated is substantially delayed. In others, the accuracy of thesystem depends on environmental conditions, such as lighting or time ofday. Therefore, in order to provide an immediate and reliable responseto a predicted occurrence of an accident, such techniques aresuboptimal.

For example, Japanese Patent Application No. 8-162911 entitled “MotorVehicle Accident Monitoring Device” (“the Japanese reference”),expressly incorporated herein by reference in its entirety, discloses asystem for monitoring traffic accidents including a plurality ofmicrophones and video cameras disposed at an intersection. Collisionsounds are chosen from among the typical sounds at an intersection. Thesource of the collision sounds is determined by comparing the timedifferences of the sounds received by each of the microphones. Imagedata from the cameras is recorded upon the occurrence of the collision.However, the Japanese reference discloses a system that is constantlyphotographing the accident scene thereby wasting video resources.

U.S. Pat. No. 6,141,611 issued to Mackey et al. entitled “Mobile VehicleAccident Data System” (“the Mackey reference”), expressly incorporatedherein by reference in its entirety, discloses an on-board vehicleaccident detection system including one or more video cameras thatcontinuously record events occurring at a given scene. Camera images ofthe scene are digitally stored after compression. An accident detectoron-board the vehicle determines if an accident has occurred, and if so,the stored images are transmitted to a remote site for observation.However, the Mackey reference includes video cameras on-board thevehicles themselves, increasing the likelihood that the cameras wouldbecome damaged during an accident thereby rendering them impractical foraccident-recording systems. Further, the on-board cameras'image-capturing ability is severely limited due to the constraints ofthe vehicle themselves. Additionally, the Mackey reference discloses asystem that determines if an accident is present by the suddenacceleration or deceleration of the vehicle, without the use of fixedmicrophones. The invention claimed by Mackey is on board the vehicle, itdoes nothing to solve the problem or record an accident in two vehicleswhich are not so equipped. Equipping every vehicle with this system isimpractical and therefore not feasible.

U.S. Pat. No. 6,111,523 issued to Mee entitled “Method and Apparatus forPhotographing Traffic in an Intersection”, expressly incorporated hereinby reference in its entirety, describes a system for taking photographsof vehicles at a traffic intersection by triggering a video camera tocapture images wherein the triggering mechanism of the video camera isbased upon certain vehicle parameters including the speed of the vehicleprior to its entrance into the traffic intersection.

U.S. Pat. No. 6,088,635 issued to Cox et al. entitled “Railroad VehicleAccident Video Recorder”, expressly incorporated herein by reference inits entirety, discloses a system for monitoring the status of a railroadvehicle prior to a potential accident. The system employs a video cameramounted within the railroad car that continuously views the status of agiven scene, and continuously stores the images of the scene. LikeMackey, it is impractical and therefore not feasible to equip everyvehicle with this system.

U.S. Pat. No. 5,717,391 issued to Rodriguez entitled “Traffic EventRecording Method and Apparatus”, expressly incorporated herein byreference in its entirety, describes a system for determining thecondition of a traffic light and includes an audio sensor which monitorssound at all times. Sound detected above a certain decibel leveltriggers the recordation of sounds, the time of day and the status ofthe traffic lights. However, Rodriguez fails to disclose video camerasor any image-capturing means.

U.S. Pat. No. 5,677,684 issued to McArthur entitled “Emergency VehicleSound-Actuated Traffic Controller”, expressly incorporated herein byreference in its entirety, describes a traffic controller systemutilizing sound detection means connected to a control box whichcontains a switching mechanism that, in a first orientation, allowsnormal operation of traffic light control and a second orientation that,upon the detection of an approaching siren, sets all traffic signals atan intersection to red to prohibit the entrance into the intersection ofadditional vehicles.

U.S. Pat. No. 5,539,398 issued to Hall et al. entitled “GPS-basedTraffic Control Preemption System”, expressly incorporated herein byreference in its entirety, discloses a system for determining if avehicle issuing a preemption request to an emergency vehicle or policecar is within an allowed approach of a traffic intersection, utilizing aGPS system.

U.S. Pat. No. 6,690,294 issued to Zierden entitled “System and methodfor detecting and identifying traffic law violators and issuingcitations”, expressly incorporated herein by reference, discloses amobile or stationary traffic monitoring system for detecting violationsof speed limits or other traffic laws by vehicle operators and issuingcitations to an operator and/or vehicle owner suspected of a violationusing a digital camera to capture images of the operator and/or thevehicle, transmitting the captured images and other relevant data to ananalysis center where the images and data are analyzed to determinewhether to issue a citation and, if so, to issue the citation or takeother appropriate law enforcement measures. The system captures imagesof a vehicle and/or vehicle operator suspected of a traffic violation,determines the time and geographic location of the suspected violation,transmits the images and other data to an analysis center, issuescitations to violators and derives revenue therefrom.

U.S. Pat. No. 5,938,717 to Dunne et al., expressly incorporated hereinby reference, discloses a traffic control system that automaticallycaptures an image of a vehicle and speed information associated with thevehicle and stores the image and information on a hard disk drive. Thesystem uses a laser gun to determine whether a vehicle is speeding. Thehard drive is later connected to a base station computer which is, inturn, connected to a LAN at which the information from the hard drive iscompared with databases containing data such as vehicle registrationinformation and the like. The system automatically prints a speedingcitation and an envelope for mailing to the registered owner of thevehicle

U.S. Pat. No. 5,734,337 to Kupersmit, expressly incorporated herein byreference, discloses a stationary traffic control method and system fordetermining the speed of a vehicle by generating two images of a movingvehicle and calculating the vehicle speed by determining the distancetraveled by the vehicle and the time interval between the two images.The system is capable of automatically looking up vehicle ownershipinformation and issuing citations to the owner of a vehicle determinedto be speeding.

U.S. Pat. No. 5,948,038 to Daly et al., expressly incorporated herein byreference, discloses a method for processing traffic violationcitations. The method includes the steps of determining whether avehicle is violating a traffic law, recording an image of the vehiclecommitting the violation, recording deployment data corresponding to theviolation, matching the vehicle information with vehicle registrationinformation to identify the owner, and providing a traffic violationcitation with an image of the vehicle, and the identity of theregistered owner of the vehicle.

The I-95 Corridor Coalition, Surveillance Requirements/Technology, Ch.4., Technology Assessment, expressly incorporated herein by reference,describes a number of different technologies suitable for incidentdetection. For example, AutoAlert: Automated Acoustic Detection ofTraffic Incidents, was an IVHS-IDEA project which uses military acousticsensor technologies, e.g., AT&T IVHS NET-2000™. The AutoAlert systemmonitors background traffic noise and compares it with the acousticsignatures of previously recorded accidents and incidents for detection.See, David A. Whitney and Joseph J. Pisano (TASC, Inc., Reading, Mass.),“AutoAlert: Automated Acoustic Detection of Incidents”, IDEA ProjectFinal Report, Contract ITS-19, IDEA Program, Transportation ResearchBoard, National Research Council, Dec. 26, 1995, expressly incorporatedherein by reference. The AutoAlert system employs algorithms whichprovide rapid incident detection and high reliability by applyingstatistical models, including Hidden Markov Models (HMM) and CanonicalVariates Analysis (CVA). These are used to analyze both short-term andtime-varying signals that characterize incidents.

The Smart Call Box project (in San Diego, Calif.) evaluated the use ofthe existing motorist aid call box system for other traffic managementstrategies. The system tests the conversion of existing cellular-basedcall boxes to multifunctional IVHS system components, to transmit thedata necessary for traffic monitoring, incident detection, hazardousweather detection, changeable message sign control, and CCTV control.

In 1992 the French Toll Motorway Companies Union initiated testing anAutomatic Incident Detection (AID) technique proposed by the FrenchNational Institute for Research on Transportation and Security (INRETS).The technique consists of utilizing computers to analyze video imagesreceived by television cameras placed along the roadway. A “mask’ framesthe significant part of the image, which typically is a three orfour-lane roadway and the emergency shoulder. The computer processesfive pictures a second, compares them two at a time, and analyzes themlooking for points that have moved between two successive pictures.These points are treated as objects moving along the roadway. If amoving object stops and remains stopped within the mask for over 15seconds, the computer considers this an anomaly and sets off an alarm.In 1993, as part of the European MELYSSA project, the AREA Companyconducted a full scale test over an urban section of the A43 motorwaylocated east of Lyons. The roadway was equipped with 16 cameras on 10meter masts or bridges with focal distances varying from 16 to 100 km,and fields of detection oscillating between 150 and 600 meters. ImageProcessing and Automatic Computer Traffic Surveillance (IMPACTS) is acomputer system for automatic traffic surveillance and incidentdetection using output from CCTV cameras. The algorithm utilized by theIMPACTS system takes a different approach from most other imageprocessing techniques that have been applied to traffic monitoring. Roadspace and how it is being utilized by traffic is considered instead ofidentifying individual vehicles. This leads to a qualitative descriptionof how the road, within a CCTV image, is occupied in terms of regions ofempty road or moving or stationary traffic. The Paris London Evaluationof Integrated ATT and DRIVE Experimental Systems (PLEIADES) is part ofthe DRIVE Research Programme. The Automatic Traffic Surveillance (ATS)system has been installed into Maidstone Traffic Control Center andprovides information on four separate CCTV images. This information willbe used both in the Control Center and passed onto the TrafficInformation Center via the PLEIADES Information Controller (PIC) anddata communications link. Instead of remote PCs there is a duplicatedisplay of the Engineers workstation that is shown in the Control Officeon a single computer monitor. The ATS system communicates data atregular intervals to the PIC. Any alarms that get raised or clearedduring normal processing will get communicated to the PIC as they occur.The PIC uses the information received to display a concise picture of avariety of information about the highway region. The ATS system usesvideo from CCTV cameras taken from the existing Control Office CameraMultiplex matrix, while not interfering with its normal operation. Whena camera is taken under manual control, the processing of the data forthat image is suspended until the camera is returned to its presetposition.

Navaneethakrishnan Balraj, “Automated Accident Detection InIntersections Via Digital Audio Signal Processing” (Thesis, MississippiState University, December 2003), expressly incorporated herein byreference, discusses, inter alia, feature extraction from audio signalsfor accident detection. The basic idea of feature extraction is torepresent the important and unique characteristics of each signal in theform of a feature vector, which can be further classified as crash ornon-crash using a statistical classifier or a neural network. Othershave tried using wavelet and cepstral transforms to extract featuresfrom audio signals such as speech signals. S. Kadambe, G. F.Boudreaux-Bartels, “Application of the wavelet transform for pitchdetection of speech signals,” IEEE Trans. on Information Theory, vol.38, no. 2, part 2, pp. 917-924, 1992; C. Harlow and Y. Wang, “AutomatedAccident Detection,” Proc. Transportation Research Board 80th AnnualMeeting, pp 90-93, 2001. Kadambe et al developed a pitch detector usinga wavelet transform. One of the main properties of the dyadic wavelettransform is that it is linear and shift-variant. Another importantproperty of the dyadic wavelet transform is that its coefficients havelocal maxima at a particular time when the signal has sharp changes ordiscontinuities. These two important properties of the dyadic wavelettransform help to extract the unique features of a particular audiosignal. Kadambe et al made a comparison of the results obtained fromusing dyadic wavelet transforms, autocorrelation, and cepstraltransforms. The investigation showed that the dyadic wavelet transformpitch detector gave 100% accurate results. One reason for the differencein the results was that the other two methods assume stationarity withinthe signal and measure the average period, where as the dyadic wavelettransform takes into account the non-stationarities in the signal.Hence, the dyadic wavelet transform method would be the best to extractfeature when the signals are non-stationary. Harlow et al developed analgorithm to detect traffic accidents at intersections, using an audiosignal as the input to the system. The algorithm uses the Real CepstralTransform (RCT) as a method to extract features. The signals recorded atintersections include brake, pile drive, construction and normal trafficsounds. These signals are segmented into three-second sections. Each ofthese three second segmented signals is analyzed using RCT. RCT is amethod where the signal is windowed for every 100 msec using a hammingwindow with an overlap of 50 msec. Thus, for a given three-secondsignal, there will be almost 60 segments of 100 msec duration each. RCTis applied to each of these segments, and the first 12 coefficients areused as the features. The features obtained using the RCT are thenclassified as “crash” or “non-crash” using a neural network.

Balraj's experimental results showed that among the three differentstatistical classifiers investigated, maximum likelihood and nearestneighbor performed best, although this had high computational costs.Haar, Daubechies, and Coiflets provided the best classificationaccuracies for a two-class system. Among the five different featureextraction methods analyzed on the basis of the overall accuracy, RCTperformed best. The second-generation wavelet method, the liftingscheme, was also investigated. It proved computationally efficient whencompared to DWT. Thus, it was concluded that the optimum design for anautomated system would be a wavelet-based feature extractor with amaximum likelihood classifier. Thus the choice of DWT or the liftingscheme would be preferred for a real-time system.

U.S. Pat. No. 6,757,574, expressly incorporated herein by reference,shows transmission of location and identification from a traffic controldevice.

U.S. Pat. No. 6,373,851, expressly incorporated herein by reference,discloses an Ethernet-connected vehicle traffic control device

U.S. Pat. No. 4,521,644, expressly incorporated herein by reference,relates to a telephone-line interconnected set of traffic controldevices, which can be remotely programmed or monitored.

In any and/or all of the embodiments described herein, the systems,equipment systems, subsystems, devices, components, and/or appliances,of and/or utilized in any of the respective embodiments, can includeand/or can utilize the teachings and/or the subject matter of thefollowing U.S. patents, the subject matter and teachings of which arehereby incorporated by reference herein and form a part of thedisclosure of this patent application: U.S. Pat. No. 6,009,356 (Monroe,Dec. 28, 1999); U.S. Pat. No. 5,890,079 (Beemer, II, et al., Sep. 7,1999); U.S. Pat. No. 5,845,240 (Fielder, Dec. 1, 1998); U.S. Pat. No.5,948,026 (Levine, Mar. 30, 1999); U.S. Pat. No. 5,446,659 (Yamawaki,Aug. 29, 1995); U.S. Pat. No. 5,056,056 (Gustin, Oct. 8, 1991); U.S.Pat. No. 6,718,239 (Rayner, Apr. 6, 2004); U.S. Pat. No. 6,449,540(Rayner, Sep. 10, 2002); U.S. Pat. No. 6,684,137 (Takagi, et al., Jan.27, 2004); U.S. Pat. No. 6,633,238 (Lemelson, et al., Oct. 14, 2003);U.S. Pat. No. 6,226,389 (Lemelson, et al., May 1, 2001); U.S. Pat. No.6,630,884 (Shanmugham, Oct. 7, 2003); U.S. Pat. No. 6,600,417 (Lerg, etal., Jul. 29, 2003); U.S. Pat. No. 6,288,643 (Lerg, et al., Sep. 11,2001); U.S. Pat. No. 6,281,792 (Lerg, et al., Aug. 28, 2001); U.S. Pat.No. 6,580,373 (Ohashi, Jun. 17, 2003); U.S. Pat. No. 6,574,548 (DeKock,et al., Jun. 3, 2003); U.S. Pat. No. 6,472,982 (Eida, et al., Oct. 29,2002); U.S. Pat. No. 5,784,007 (Pepper, Jul. 21, 1998); U.S. Pat. No.6,466,260 (Hatae, et al., Oct. 15, 2002); U.S. Pat. No. 6,429,812(Hoffberg, Aug. 6, 2002); U.S. Pat. No. 6,252,544 (Hoffberg, Jun. 26,2001); U.S. Pat. No. 6,389,340 (Rayner, May 14, 2002); U.S. Pat. No.6,324,450 (Iwama, Nov. 27, 2001); U.S. Pat. No. 6,211,907 (Scaman, etal., Apr. 3, 2001); and U.S. Pat. No. 5,689,442 (Swanson, et al., Nov.18, 1997); U.S. Pat. No. 6,647,270 (Himmelstein, Nov. 11, 2003); U.S.Pat. No. 6,574,538 (Sasaki, Jun. 3, 2003); U.S. Pat. No. 6,573,831(Ikeda, et al., Jun. 3, 2003); U.S. Pat. No. 6,542,077 (Joao, Apr. 1,2003); U.S. Pat. No. 6,404,352 (Ichikawa, et al., Jun. 11, 2002); U.S.Pat. No. 6,401,027 (Xu, et al., Jun. 4, 2002); U.S. Pat. No. 6,392,692(Monroe, May 21, 2002); U.S. Pat. No. 6,339,370 (Ruhl, et al., Jan. 15,2002); U.S. Pat. No. 6,314,364 (Nakamura, Nov. 6, 2001); U.S. Pat. No.6,163,338 (Johnson, et al., Dec. 19, 2000); U.S. Pat. No. 6,154,658(Caci, Nov. 28, 2000); U.S. Pat. No. 6,091,956 (Hollenberg, Jul. 18,2000); U.S. Pat. No. 6,087,960 (Kyouno, et al., Jul. 11, 2000); U.S.Pat. No. 6,075,466 (Cohen, et al., Jun. 13, 2000); U.S. Pat. No.5,990,801 (Kyouno, et al., Nov. 23, 1999); U.S. Pat. No. 5,943,428(Seri, et al., Aug. 24, 1999); U.S. Pat. No. 5,699,056 (Yoshida, Dec.16, 1997); U.S. Pat. No. 5,353,023 (Mitsugi, Oct. 4, 1994); U.S. Pat.No. 5,025,324 (Hashimoto, Jun. 18, 1991); U.S. 20040022416 (Lemelson, etal., Feb. 5, 2004); U.S. 20020008637 (Lemelson, et al., Jan. 24, 2002);U.S. 20030225516 (DeKock, et al., Dec. 4, 2003); U.S. 20020193938(DeKock, et al., Dec. 19, 2002); U.S. 20030222981 (Kisak, et al., Dec.4, 2003); U.S. 20030214405 (Lerg, et al., Nov. 20, 2003); U.S.20020008619 (Lerg, et al., Jan. 24, 2002); U.S. 20030125853 (Takagi, etal., Jul. 3, 2003); U.S. 20030081935, 20030081934, 20030081128,20030081127, 20030081122, 20030081121, and 20030080878 (Kirmuss, May 1,2003); U.S. 20020121969 (Joao, Sep. 5, 2002); U.S. 20020147982 (Naidoo,et al., Oct. 10, 2002); U.S. 20030062997 (Naidoo, et al., Apr. 3, 2003);U.S. 20010005804 (Rayner, Jun. 28, 2001); U.S. 20020163579 (Patel, etal., Nov. 7, 2002); U.S. 20020170685 (Weik, et al., Nov. 21, 2002); U.S.20030011684 (Narayanaswami, et al., Jan. 16, 2003); U.S. 20030041329(Bassett, Feb. 27, 2003); U.S. 20030053536 (Ebrami, Mar. 20, 2003); andU.S. 20030067542 (Monroe, Apr. 10, 2003).

The following references are incorporated herein by references as ifexplicitly set forth herein:

www.itsdocs.fhwa.dot.gov/JPODOCS/REPTS_TE/36D01!.PDF;www.ndsu.nodak.edu/ndsu/ugpti/MPC_Pubs/html/MPC01-122.html;www.intelligenthighway.com/ITS/IMITS.pdf;stat-www.berkeley.edu/users/kwon/papers/inc_detection.pdf;www-users.cs.umn.edu/~masoud/publications/harini-intersection-itsc-2002.pdf;

The I-80 Experiment: Real-Time Algorithms for Travel Time Estimates andIncident Detection—Alex Skabardonis;

Dougherty, M. S., Chen, H. and Montgomery, F. (1998), Principlecomponents based incident detection Proc. 5th International Conferenceon Applications of Advanced Technologies in Transportation, NewportBeach, Calif., ieeexplore.ieee.org/xpl/tocresult.jsp?is Number=14013;

Karim, A. and Adeli, H. (2003), “Fast Automatic Incident Detection onUrban and Rural Freeways Using the Wavelet Energy Algorithm,” Journal ofTransportation Engineering, ASCE, Vol. 129, No. 1, pp. 57-68.www.pubs.asce.org/WWWdisplay.cgi?0300116;

Chen, H; Boyle, R D; Montgomery, F O; Kirby, H R. Novelty detection formotorway incident management in: Proceedings of the Workshop onEngineering for Intelligent Systems. 1998.portal.acm.org/citation.cfm?id=643217&d1=ACM&col1=GUIDE;

Karim, A. and Adeli, H. (2002), “Comparison of Fuzzy Wavelet RadialBasis Function Neural Network Freeway Incident Detection Model withCalifornia Algorithm,” Journal of Transportation Engineering, ASCE, Vol.128, No. 1, pp. 21-30, www.pubs.asce.org/WWWdisplay.cgi?0200117;

Chien-Hua Hsiao, Ching-Teng Lin, and Michael Cassidy, “Application ofFuzzy Logic and Neural Networks to Automatically Detect Freeway TrafficIncidents”, Journal of Transportation Engineering, Vol. 120, No. 5,September/October 1994, pp. 753-772;

A. Samant & H. Adeli, “Feature Extraction for Traffic Incident DetectionUsing Wavelet Transform and Linear Discriminant Analysis”,Computer-Aided Civil and Infrastructure Engineering, Volume 15 Issue 4Page 241 (July 2000);

Hojjat Adelil and Asim Karim, “Fuzzy-Wavelet RBFFN Model For FreewayIncident Detection”, Journal of Transportation Engineering, Vol. 126,No. 6, November/December, 2000;

H. Veeraraghavan, O. Masoud, N. P. Papanikolopoulos, “Computer visionalgorithms for intersection monitoring”, IEEE Trans. on IntelligentTransportation Systems, vol. 4, no. 2, pp. 78-89, June 2003;

K. Stubbs, H. Arumugam, O. Masoud, C. McMillen, H. Veeraraghavan, R.Janardan, N. P. Papanikolopoulos, “A real-time collision warning systemfor intersections”, in Proc. ITS America 13th Annual Meeting,Minneapolis, Minn., May 2003;

H. Veeraraghavan, O. Masoud, N. P., Papanikolopoulos, “Vision-basedmonitoring of intersections”, in Proc. IEEE 5th International Conferenceon Intelligent Transportation Systems, pp. 7-12, Singapore, September2002;

Petty, K., Ostland, M., Kwon, J., Rice, J. and Bickel, P. (2002) “A NewMethodology for Evaluating Incident Detection Algorithms,”Transportation Research, Part C, Volume 10, pp. 189-204;

Peter T. Martin, Joseph Perrin, Blake Hansen, Ryan Kump, Dan Moore,“Incident Detection Algorithm Evaluation” (University of Utah, Preparedfor Utah Department of Transportation) March 2001; and

Dastidar, Samanwoy & Adeli, Hojjat (2003), “Wavelet-Clustering-NeuralNetwork Model for Freeway Incident Detection.”, Computer-Aided Civil andInfrastructure Engineering 18 (5), 325-338.

Wireless local area networks and wide area networks are well known, andare the subject of many accepted standards, such as encompassed by theIEEE-802.11 standards and proposed standards, including for example802.11a, 802.11b, 802.11h, 802.11R/A (DSRC), as well as various IETFRFC's. These will not be discussed herein in detail, but are well knownto those of ordinary skill in the art, and it is understood that thecontent of these is incorporated herein by reference as if fully setforth herein.

SUMMARY OF THE INVENTION

Intelligent Transportation systems typically require distributedelectronics systems physically located near the transportationinfrastructure. For example, sensors, controls and communication systemsare typically not fully centralized. Environmentally sealed enclosuresare well known for this application, for example NEMA-3, 3R, 3S, 4, 4X,6 and 6P.

See,http://www.nema.org/index_nema.cfm/1427/B9A5681E-0C0C-4248-A032BE08D6B002B3/

This physical infrastructure provides a unique opportunity to providenearly seamless coverage of populated areas with wireless local areanetwork or wide area network coverage, supporting both stationary andmobile transceivers, particularly in areas with a high concentration ofintersection and traffic control devices. Further, through integrationof the access point into the intelligent transportation system, varioustelematics and intelligent highway control applications are supported.Thus, since available wireless local area networks (WLAN) providerelatively high bandwidth at low cost, entertainment applications mayalso be supported.

The WLAN and ITS integration provides a unique opportunity for vehicularcoordination and traffic intersection control, since directcommunications in real time between vehicles, public safety or emergencypersonnel, and the ITS infrastructure is facilitated. For example,traffic light timing may be dynamically adjusted for maximum efficiencyand/or public safety, and enhanced communications directed atcooperating vehicles may provide enhanced control and warning options,and, for example, the ITS may provide a visual indication (e.g.,transmitted video) representing prospective conditions that the vehiclemay encounter or seek to avoid.

Examples of applications that can be incorporated into this designinclude providing seamless wireless Internet connectivity to an entirearea, locating and tracking vehicles, people or objects equipped withtransponders, and components of traffic information systems to providetraffic information which is relayed or sent directly to subscribers.Some of these applications may be incorporated into a main system, likean accident detection and reporting system, where a power source andother components are readily available, and lend themselves readily tointegrating other components. Applications that lend themselves tocharging of user access fees may even be used to defray the cost of themain system.

Typically, the controller for an ITS system is not a personal computer,nor does it operate using a graphic user interface (GUI) shell oroperating system for its normal operation, although for diagnostics andsetup, a GUI may be provided. Thus, while the electrical and low-levelcommunications standards may be shared with personal computers, thehigher level protocols and application programming interfaces (APIs) maydiffer. The ITS system is typically a real time system, and thereforeessentially non-deterministic operating systems are avoided. On theother hand, if the level of non-determinism can be limited to anacceptable worst case, then it is possible to use such an operatingsystem. For example, both Windows XP/CE and Linux both have variantswhich are intended to be deployed in real time environments. In anycase, the operating system for the main system may be transparent to theoptional module, except where the module comprises software to beexecuted in the main system processor.

In fact, an important aspect of the ITS with WLAN is that by employingstandards-based design, it is not necessary to now contemplate orunderstand all of the potential uses for the system. Clearly, thosefunctions which are desired may be made a standard part of the system.Rather, it is an important part of a preferred embodiment of theinvention to provide a distributed Intelligent Transportation Systeminfrastructure which is not particularly functionally limited by theinitial design decisions made during original manufacture, and to allowopen or proprietary additions to the system after-the-fact.

The present invention provides a physical environment, electrical andlogical interface for at least one module within an enclosure locatednear a roadway, as part of an intelligent transportation systemdistributed infrastructure. Preferably, the module is optional, andmakes use of material resources of the main system collocated within theenclosure particularly its power supply, communication link and in someinstances its location information. Therefore, one or more sensors,traffic control device controllers, and communications facilities may beshared between the main system and the optional system, leading toeconomies and higher integration. Typically, at a minimum, a computerreadable software program storage media is provided in the module, withall hardware resources provided directly by the main system. However, inthis case, the functionality might more efficiently be directlyintegrated in the main system without the segregation, which, forexample, comprises an electrical connector or slot which allows removalof the module. More commonly, the module also comprises specifichardware which is not a normal part of the main system, which isemployed in fulfilling the function of the module. For example, themodule may comprise a communications system, a sensor interface system,or a processor or co-processor.

Accordingly, the module may be contained in an external environmentalenclosure and connected to the main system through an externalinterface, or it may be contained inside the enclosure of the mainsystem, utilizing its resources through a similar interface inside themain system enclosure.

Examples of applications that can be incorporated into this modulardesign include providing seamless wireless Internet connectivity to anarea proximate to the system, locating and tracking vehicles, people orobjects equipped with transponders, and components of trafficinformation systems to provide traffic information which is relayed orsent directly to subscribers. Some of these applications may beincorporated into a main system, like an accident detection andreporting system, where a power source and other components are readilyavailable, and lend themselves readily to integrating other components.Applications that lend themselves to assessment of user access fees mayeven be used to defray the cost of the main system.

The enclosure of the main system may accommodate one or more modules.Each module is designed to fit within a specified space, and having astandardized mechanical interface. For example, the module may meetspecifications for an ISA, EISA, MCA, PCI, PC-Card (PCMCIA) or otherhome or office computer-based standard. The module might otherwisecorrespond to a VME or other industrial computer form factor standard.In the above-identified standards, a basic electrical interface isspecified, and further a communications protocol defined as well. Inother cases, a separate electrical and/or communication protocol may beemployed, for example USB 1.0, USB 1.1, USB 2.0, IEEE-1394, IEEE-1488,RS-232, RS-423, SCSI, SCSI-II, Fast/wide/fast+wide SCSI, IEEE-802.x(thin Ethernet, thick Ethernet, Token Ring, etc.), T-1, ISDN, DSL, ADSL,DOCSIS, ATA, SATA, SCA, SCA-2, etc.

Typically, the controller for an ITS system is not a personal computernor does it operate using a graphic user interface (GUI) shell oroperating system for its normal operation, although for diagnostics andsetup, a GUI may be provided. Thus, while the electrical and low-levelcommunications standards may be shared with personal computers, thehigher level protocols and application programming interfaces (APIs) maydiffer. The ITS system is typically a real time system, and thereforeessentially non-deterministic operating systems are avoided. On theother hand, if the level of non-determinism can be limited to a worstcase, then it is possible to use such an operating system. For example,both Windows XP/CE and Linux both have variants which are intended to bedeployed in real time environments. In any case, the operating systemfor the main system is typically transparent to the optional module,except where the module comprises software to be executed in the mainsystem processor.

The enclosure is typically environmentally sealed, eliminating arequirement for the module to have its own environmental sealing, andpermitting use of unsealed or card-edge connectors, although a modulecan also be contained in an environmentally enclosure, and designed toconnect externally to the module of the main system which then hasappropriate connections to accommodate the external module.

In one embodiment, the module meets the size and electrical interfacerequirements for a blade server, for example designed to fit within astandard rack mount, and employing standard power and communicationsinterfaces. The module can be specified to fit in a 7U height EIA-310-Dstandard 19″ rack, with full, ½, ⅓, ¼, ⅙, or 1/12 width (includingrequired clearances).

In cases where the enclosure is small, for example in the case of acable-suspended enclosure, correspondingly smaller form factors may beemployed, such as PCI and cardbus. A logical module (defined by itsconsolidated function) may span one or more physical slots orconnectors.

According to a preferred embodiment, data communications between themodule and main system employ IEEE-802.3u interface using TCP/IPprotocol. This interface provides standardized connectors,communications, and symmetry. While it is non-deterministic, in acontrolled environment with peak demand well below capacity, thedeviations from ideal are small. This also allows communications withsensors using the same interface, and allows the sensors and otherresources to be shared between the main system and modules. OtherEthernet systems, such as 802.11x may also be bridged. Further, thisalso facilitates communications external to the enclosure, although itis preferred that communications occur through a firewall and, if on ashared physical network such as the Internet, that a virtual privatenetwork (VPN) be implemented.

A preferred embodiment of the MITS comprises an environmentally hardenedsingle board computer design that does not require a cooling fan for theprocessor, and which is operable over the range of environmentaltemperatures typically encountered in traffic control device enclosures.Non-mechanical cooling, such as heat pipes, Peltier coolers, or thelike, may be employed as necessary.

The single board computer (SBC) may comply with, for example, PC/104,PC/104 Plus, EPIC™ CompactPCI, STD 32, ETX, VME, ESM, or EBXspecifications. The SBC may operate under various operating systems,including for example, Linux or variants thereof, Windows XP, WindowsCE,and VxWorks®. The module may also include so-called “blade servers” andassociated components, available from various vendors according tovarious specifications. In general, the MITS will have line poweravailable, and may therefore include a temperature regulated heater toprevent lower temperature extremes from limiting system function.However, it is preferred that the MITS be capable of operating atelevated temperatures without environmental control. One acceptable SBCis a Micro/sys (Montrose, Calif.) SBC 4495-1-ET, which is an extendedtemperature range EPIC standard SBC, which comprises a 486DX processor,GPS (44950PT19), digital and analog I/O, 10/100 BaseT Ethernet,CompactFlash connector, and PC104 bus interface. Alternately, the MSEBXfrom DIGITAL-LOGIC AG, (Luterbach CH-4542), www.digitallogic.ch, whichsupports audio and video inputs directly, may be employed. The storagefor programs and data to be preserved is preferably in CompactFlashmemory, and the system may have 512 MB-8 GB of storage capacity, forexample. As memory density increases and costs decrease, far greaterstorage capacities will, of course, be feasible. Also available, but notpreferred as system data busses, include ISA, EISA, MCA, PCI, and othercommon personal computer bus architectures not particularly intended forenvironmentally hardened applications.

The enclosure for the SBC and optional hardware is preferablyenvironmentally sealed against weather, and preferably provides at leastone open slot for hardware expansion. Preferably, the enclosure has acapacity to add at least two additional system bus-based expansioncards, beyond the basic SBC. In this manner, added hardware need notemploy redundant hardware features, and may employ the resources of theSBC and existing peripherals. One expansion card which may be ofparticular use is an acoustic signature detection and analysis card,which employs, for example, one or more digital signal processors.Another expansion card which may be of use is a digital videocompression card, for example compressing video streams captured at thescene into an MPEG-4 format. If cryptographic processing burdens arebeyond the capabilities of the SBC, a cryptographic coprocessor card maybe provided. Video analysis, and in particular video scene simulation toprovide a two-dimensional model of the scene or location based on inputsfrom multiple video camera sources and optionally other sensors, may beperformed on a separate optional card. This video scene analysis allows,for example, the agent and the remote monitoring center to receive asingle video representation of conditions at the scene, synthesized torepresent a virtual view, such as looking down, or interactively definedby the agent, without having to transmit all of the raw video data inrealtime. Where relevant, the raw video data (in digitally compressedform, and cryptographically authenticated), may be stored locally untilit is either no longer needed or successfully transferred to the remotelocation, as confirmed by a message from the remote location.

Because of the public safety issues raised by the IntelligentTransportation System infrastructure, it is generally preferred that anyadditions or modifications to the system, either hardware or software,be certified by a certification authority for function, compatibility,and security, prior to deployment. Likewise, the MITS preferablyincludes an available means for updating and patching, as updates areproduced, vulnerabilities discovered and remediated, or where changes tosystem or optional software are desired. These communications can occurthrough a satellite radio link, similar to XM Radio or Sirius, wired orwireless terrestrial network, or the like. For example, one link to eachMITS may include an IEEE-802.16x (WiMax), digital subscriber line (DSL)or 3G cellular radio for data communications, which would allowcommunications directly to each module. In each case, it is preferredthat a more reliable backup system be available, since WiMax, DSL andcellular services are subject to various quality of service impairmentsand untariffed repair limitations. The backup system may be, forexample, an ad hoc radio network (mesh network), for example using thevarious MITSs as dispersed nodes, using licensed public safety spectrum.The ad hoc network is, for example, a multihop network, or other knownscheme. The network may include or exclude mobile nodes, and may haveredundant communications paths using the same or different physicaltransport layers (e.g., various types of optical, wired or wireless).Permanent use of public safety licensed spectrum by the MITSs may beimprudent, since this might consume valuable bandwidth which mightotherwise be available for other uses.

One embodiment of the present invention provides, as a basic function ofthe MITS, is a vehicle accident detection and data recordation andtransmission system that places one or more video cameras, microphonesand data collection and transmission apparatus in proximity to trafficintersections, or other desired locations, to monitor local conditions,and, for example, to detect, store, and archivally retainaccident-related images and sounds, together with other accident-relateddata such as time and location, and to relay data to real-timemonitoring centers, where the information can be reviewed immediatelyfor the purpose of screening false alarms, assessing the severity of theaccident and dispatching an appropriate level of emergency response. Tothis end, one aspect of the present invention provides business modelsfor financing at least a portion of the system by imposing usage feesfor to access to communications resources and/or real-time or archivaldata obtained from one or more of the systems

According to the present invention, a system is provided to monitor thesounds at a traffic intersection (or other location where monitoring isdesired), such that when certain sounds are detected that indicate anautomobile incident (such as an accident) is imminent or is in process,the system records the audio, video and other information pertinent tothe incident such as location, time, state of the traffic controlsignals (if any and if desired), and transmits the data to a remotecontrol center where the state of the accident scene can be assessed, anappropriate response dispatched to render assistance, and the accidentrelated data can be archived for later use in assessing fault andliability by the authorities, the courts and the insurance companiesrepresenting the parties to the accident for assessing. The location andtime of the accident detection are determined with a high degree ofaccuracy, for example, by using a satellite navigation system receiversuch as the existing Naystar Global Positioning System (GPS) currentlyin use by the United States government. To alleviate the need for anymajor modifications to the existing traffic control infrastructure, thesystem preferably uses existing wireless systems or networks, such ascellular (2G, 2.5G, 3G, etc), WLAN (IEEE 802.11x), direct broadcasttransmission, ad hoc (mesh) networks, microwave or laser transmission,or other type communications, to transmit the accident data, andutilizes existing monitoring services as control centers to receive andprocess the accident. The basic hardware components of the invention arecommercially available, although dedicated, customized, and/or highlyintegrated systems may also be made for this purpose. By providingimmediate notification of accident conditions, as well as live or nearreal-time video feeds, public safety officials are provided withenhanced tools, and public safety is enhanced. Further, the presentinvention provides enhanced homeland security, by providing improvedmonitoring of the public infrastructure.

A particular advantage of a preferred embodiment of the presentinvention is that data screening is provided prior to transmission,based on an intelligent analysis of the environment, including logicalanalysis and heuristics. By providing appropriate filtering of thefeeds, as well as the capability to transmit raw data, or relativelyunprocessed feeds, a human operator can assess the situation. This humandata presentation aspect means that the error threshold may be set at alevel which minimizes or eliminates the false negatives, while limitingthe false positives to an acceptable level. Therefore, the humanmonitors can be used efficiently.

The present system and method will therefore save lives and improvepublic safety by facilitating almost instant reporting of trafficaccidents or other events on streets and intersections and locations soequipped, and will save time and money of the part of the authorities,courts, insurance companies and the accident victims by creating anaudio and video record of the accident which can be use to determinefault and liability. Other potential benefits to society includeminimizing the dispatching of emergency medical response teams toincidents where they are not needed, thereby leaving these resourcesmore readily available for true emergencies, and a reduction in theburden on the judicial system, as the parties to an accident and theirrepresentatives will have undisputable evidence of fault makingout-of-court settlements more likely.

The present system also permits monitoring of various locations bycentralized monitoring centers, or even by persons seeking the data,which would not be considered highly confidential. That is, if a driverwishes to investigate the traffic at a particular intersection, he coulduse a video-enabled phone, such as a video-conferencing cellular phone,to communicate with the monitoring device (or more likely, with a serversystem which communicates with the monitoring device, to allowmulticasting and arbitration of access, as well as cost accounting), toview and/or listen to conditions at the monitored location. Of course,in sensitive situations, data encryption and/or user authentication maybe provided to secure the data stream.

The ability for the public to access the location monitoring system dataprovides a means for subsidy of the deployment of the system, forexample through a subscription, pay-per-use, or advertising-subsidymodel. Thus, the cost impact on the agency may be blunted, whilepermitting a longer-term view of the costs and benefits of the system.The agency can also assess the at fault party with a fine or charge,assessing the costs of implementation of the system on those whodirectly benefit or are found liable for an incident detected. Theincident records may be used to support imposition of the fee. Theagency may also impose an access fee for the data. The system is alsosufficiently flexible as to enable alternate embodiments to be adaptedto include ancillary uses, such as traffic signal and speed enforcement.Adding such features has the potential to generate additional revenuefor agencies operating the invention, while potentially improvingtraffic safety which should in turn help to minimize the number ofaccidents.

The ability to initiate a stream from a location monitoring systemgenerally arises from the use of a standard communications system,rather than a dedicated and/or proprietary communications system.Therefore, it is preferred that the location monitoring systemcommunicate over public communications infrastructure, such as cellular,wired telephone/DSL/WiMax/Cable modem, Internet, unlicensed spectrumusing industry standard protocols, licensed spectrum using any ofvarious modulation schemes and protocols, or the like. Of course, theuse of such public communications infrastructure is not required. It isalso optional for the location monitoring system, especially for publicsafety applications, to have a backup communications system, so that inthe event of a failure or interference, the system remains operational.Preferably, when used, the redundant systems operate through a differentphysical communications layer, and are thus subject to different typesof interference and modes of failure.

A preferred embodiment incorporates one or more sound capturing devicesand one or more image-capturing devices connected to a control unit tolisten for accident related sounds and to capture audio and video imagesof an accident. The control unit contains Random Access Memory (“RAM”)and data processing and storage capabilities for processing and storingaudio, video, location, time and other accident related data such as thestate of any traffic signals at the time of the accident if any arepresent, and for communicating with and accepting command and controlfrom a remote location. Also contained within or connected to saidcontrol unit are a satellite navigation system receiver or other meansfor capturing, recording and reporting the location and time of anaccident, and a means for communicating with a remote location which canbe a wireless transceiver, wired or wireless network connection or adirect connection to the Public Switching Telephone Network (“PSTN”).The communication means is also used by the control unit for initiatingcontact with a remote location for the purpose of reporting andtransferring accident related data to the designated remote location,and for receiving command and control signals from said remote location.A particular advantage of using a GPS geolocation system is that itprovides accurate location and time data, while alleviating the need toprogram the location monitoring device with identification or locationdata, or to track the identification of each location monitoring deviceat a central station. The devices are therefore self-registering basedon their reported accurate location, facilitating installation,maintenance, and service.

The control unit and its components together with sound andimage-capturing devices connected to (or contained within) said controlunit are positioned proximate a desired location such as trafficintersection or busy street. Acoustical data received from the soundcapturing devices is processed in the control unit to determine if thoseacoustical sounds meet predetermined threshold levels or signaturepatterns that indicate an accident is about to occur (“preliminarysounds”) or is in the process of occurring (“qualifying sounds”). In thepreferred embodiment, the control unit uses RAM or other data storagemeans as a buffer, and continually stores in the buffer all audiosignals and video images of the desired location in a loop or circularbuffer that retains data for a specified period of time, overwritingaudio and video that exceeds the specified period of time. Of course, itis also possible to continuously record the data or stream it from themonitoring device, though this is not necessarily efficient. Thetemporary storage system or buffer may include dynamic random accessmemory, static random access memory, persistent electricallyprogrammable and erasable memory of various kinds (EEPROM, Flash,ferroelectric, etc.), rotating magnetic media, magnetic tape recordingmedia, rewritable optical storage media, magneto-optical media,holographic storage media, or the like. Non-rewritable memory may alsobe used to form a permanent archive of various events.

When a qualifying sound is detected, the system stops overwriting oldinformation in the circular buffer, thereby saving audio signals andvideo images leading up to the qualifying sound, and continues savingsubsequent audio and video until the control unit is reset. The data is,for example, transferred from the circular buffer to a persistentstorage device. In this embodiment, the system is not dependent onpreliminary sounds, and is designed to capture the events leading up toan accident.

In the event that communications with the monitoring center areunavailable, the data is preferably retained locally until retrieved.Since secondary accidents are common, it is preferred that the systemcontinue to monitor and/or record data from the accident or event scenefor some time after initial triggering.

In another embodiment, preliminary sounds can be used to start recordingof audio signals and video images. These alternate embodiments do notnecessitate continually storing images leading up to a qualifying soundas all audio and video signals following a preliminary sound are stored.In these alternate embodiments, when preliminary sounds are detected,the control unit begins storing audio signals and video images of thedesired location (“the accident scene”) in the RAM or data storagemeans. When qualifying sounds are detected within a predetermined amountof time after detection of preliminary sounds, the control unitcontinues storing audio signals and video images of the accident sceneand also stores the time and location data from the satellite navigationreceiver or other means for determining time and location, and thewireless transceiver or other communication means initiates contact withthe designated remote location (“the monitoring center”). If qualifyingsounds are detected without being preceded by preliminary sounds, thenthe control unit begins storing all audio, video, location, time andother accident related data, and initiates contact with the monitoringcenter immediately.

If a qualifying sound is not detected within a predetermined amount oftime after a preliminary sound is detected, the stored audio and videosignals that followed the preliminary sound may be discarded and thecontrol unit resumes waiting for the next preliminary or qualifyingsound to be detected.

The preferred embodiment therefore allows deferred processing of thesensor data, and allows decisions to be made after more completeinformation is available. For example, after a preliminary sound isdetected, instead of focusing on the qualifying sound, the video datamay be analyzed for evidence of an accident. A particular characteristicof a collision is a rapid deceleration. This can be detected in a videoscene, for example, by analyzing motion vectors. However, without theaudio analysis, the video analysis alone might produce many falsepositives, which would limit the ability of a small number of humanagents at a central monitoring center to handle a large number of remotesensing systems.

When contact with the monitoring center is established after aqualifying sound is detected, the control unit transmits the locationand still or moving video images of the accident scene which aredisplayed, for example, on a video monitor at the monitoring center. Thedetermination of whether to use still or moving images at this step maybe preprogrammed into the control unit according to predetermined userpreferences which may be determined in part by the available bandwidthof the communications means being utilized, and the preferences of theagency implementing the system. In general, the data will be presentedto monitoring agents in a standardized format. It is also possible togenerate a synthetic view of the scene for an agent, for example byprocessing and combining data from a plurality of sensors into a singledisplayable presentation. For example, the standard view may be anoverhead view without parallax. The view may be generated by combiningvideo data from one or more video cameras, and processing the data toproject it into the desired framework. Audio data may also be processedinto a standard format, regardless of where the microphones are located.

The person at the monitoring center (“the operator”) can then determinethe location of and assess the accident scene, notify the properauthorities and relay the information needed by said authorities so theycan dispatch the appropriate emergency response. Such information mayinclude the number of vehicles involved, potential injuries, presence offire, severity of the wreckage, obstruction of traffic, all of which canhelp the authorities dispatch the appropriate response and determine thebest direction from which to access the accident scene. Further, the rawdata, from the original incident and also in real time, may be madeavailable to the authorities for analysis and location scene management.In some embodiments, it may be desirable to enable the operator tomanage the traffic signals at the accident scene to facilitate access toemergency vehicles. Instead of using an already existing monitoringcenter, it may be desirable for the agency to implement its ownmonitoring center or integrate the system into an existing dispatchingsystem.

The stored audio signals, video images, time and location data and otherdata about the accident scene such as the state of the traffic lights(“accident-related data”) is then transmitted to and saved at themonitoring center or another remote location so as to create a permanentrecord of the accident-related data. When the accident-related data hasbeen successfully transmitted and permanently stored, a command orconfirmation signal may be sent to the control unit that resets thecontrol unit, and permits the connection to be closed, if appropriate.For example, the command may instruct the RAM and data storage means tobe cleared and reset. While the raw data is continuously monitored, theanalysis may proceed in stages. After “reporting” an incident, thecontrol unit may then revert to its normal monitoring and analysismodes, e.g., detecting of preliminary or qualifying sounds depending onthe embodiment.

The communication means in the control unit is also used for command andcontrol in order to program and managed the control unit remotely,perform diagnostics and troubleshooting, and to otherwise manage thecontrol unit and its components from a remote location such as themonitoring center or other remote facility. Security means can be usedto prevent unauthorized access to the command and control programming ofthe control unit. Such means may include password or cryptographicaccess restriction, channel and/or user authentication, and/orphysically (private network and/or unshared physical communicationchannel) or logically (virtual private network) closed communicationsystems. The security system may also encompass a so-called “firewall”which inspects various characteristics of a communication over a sharedphysical communication channel and grants or denies transfer of theinformation accordingly. The security system may therefore completelylimit access, limit modification or alteration of settings, such ascommand and control settings, or stored data representing the forensicevidence to be preserved and authenticated, or some combination of theabove. Protection of the data content against tampering is preferably byboth physical and cryptographic processes, wherein the data iscryptographically authenticated for both time of acquisition and contentat or near the time of creation, in a manner where exact recreation isnearly impossible. The various times may be relevant to the operation ofthe system and use of the resulting data. Typically, each image will beassociated with a timecode, that is, a code representing the time(absolute or relative) the image was created, which will normally becommunicated with the images or video signal. Typically, there will bevarious timecodes, including those associated with images, but possiblyalso without associated images, such as a time of relevant trafficcontrol device changes (such as the time a traffic light turns red), atime of detection of an acoustic signal representing a preliminary soundanticipating a vehicular incident or non-vehicular incident, a time of adetermination that a vehicular or non-vehicular incident has occurred,or other times. Since a portion of the data to be transmitted to theremote location is not transmitted in real time, it is clear thattransmitted timecodes in non-real time data will differ from an actualtime of transmission. It is also clear that there will be minutedifferences between the actual time of the sounds leading up to suchdetection and determination, and the time of such detection anddetermination, as there will be a lag between the time of the sound andthe time it is received and processed. While the differences arenegligible, it is possible to determine the actual time of an imminentor actual incident, and the state of the traffic control device at suchtimes, by correlating the time of acoustic data with correspondingimages (for example, a given image with a time stamp may show an actualcollision fractions of a second before it was detected). In the case ofreal time transmissions, the criticality of including timecodes isdiminished, since these can be recreated on receipt. On the other hand,in order to detect tampering of transmissions, the use of such timecodesmay be important, and a comparison of a transmitted timecode with ananticipated timecode may be useful. While a current time may bedetermined based on a free-running clock, advantageously, the precisetime may be extracted from a satellite or network signal, since in apreferred embodiment, satellite and/or network data feeds arecontinuously available. In particular, since GPS technology is a timedependent, a very precise clock is available as part of a GPS receiver.

The system may acquire data representing both vehicular andnon-vehicular incidents. One example of a non-vehicular incident is agunshot. Because the acoustic sensors can readily acquire acousticsignatures indicative of a gunshot, and a plurality of acoustic sensorscan triangulate the location of a gunshot, a reporting of this type ofincident, especially in conjunction with video representing the scene atthe time of detection, may be very useful, especially in high crimeareas. Further, because the timing of events can be determined veryaccurately based on GPS-associated time-codes, the acoustic sensors froma plurality of locations can be used to determine a location of theevent, even if it is somewhat distant or out of line-of-sight.Preferably, this multi-location sensing is coordinated at a remotemonitoring center, although direct communications between locations mayalso be used to coordinate analysis and reporting.

Another application of the preferred system for non-vehicular reportingis explosion or bomb blast incidents. While such occurrences arerelatively rare, the analysis and reporting is available withoutsubstantial hardware adaptation, and thus may be implemented usingsoftware in a basic vehicular incident detection system. Further,because the bomb blast or explosion acoustic signature would requireanalysis by the system and distinguishing from vehicular incidents, theincremental efforts required to preserve data and report the event wouldnot be a significant burden on the system.

The control unit and other components of the system may also contain orbe attached to backup batteries to provide power in times of electricalfailure. When used, the preferred method for keeping these backupbatteries charged is by direct electrical connections, although solarmeans or other means for keeping batteries charged may be employed. Inalternate embodiments where the sound-capturing means andimage-capturing means are connected to the control unit by wirelessmeans, those devices can also be equipped with backup batteries.

Typically, the control unit will be mounted on or near traffic signals,thus providing a good vantage point, access to power, and relativefreedom from vandalism.

Specifically, a preferred embodiment of the present invention provides asystem for determining the occurrence or imminent occurrence of anautomobile accident at a given location such as a traffic intersectionor busy street, and for capturing and processing relevantaccident-related data including audio, video, time, location and trafficsignal information if present, and for communicating with andtransmitting the accident-related data to a remote location which may bethe proper authorities or a facility capable of notifying the properauthorities, and to create a permanent record of the accident relateddata which can be used to determine the cause of the accident, assessfault, and used as evidence in any subsequent legal proceedings.

In the preferred embodiment, the control unit contains random accessmemory (“RAM”), data processing means such as one or moremicroprocessors and other circuitry needed for the components of thesystem to function, and a hard drive or other non-volatile storagemedium for persistent data storage, in a self-contained housing. The RAMis used to capture and temporarily store acoustical, video andaccident-related data, command and control signals, and interface tooperate the components of the system. The hard drive or other storagemedium is used to store accident related data, command and controlsignals, and programming for the system. The data processing meanscontrols the function of the system and its components as explained inmore detail below. In alternate embodiments, programming for the systemcan be maintained in the data processing means and accident-related datacan be stored exclusively in the RAM memory or in place of a hard drive,accident related data can be saved on one of many possible storage meansincluding optical and tape drives, flash memory or other data storagemeans currently in use or which may be invented in the future, theobject being to have the capability of storing data includingaccident-related data and command and control signals and programming.In yet other alternate embodiments, in place of RAM alternative datastorage means such as flash memory may be utilized to temporarily storethe acoustical signals, video images, other accident related data andcommand and control signals.

It is understood that, while in a preferred embodiment, the filtering ofthe data stream occurs within the control unit, that in alternateembodiments that data may be transmitted for remote analysis. However, acommon feature of both these embodiments is that the data is filteredbefore presentation to a human agent as part of an accident managementsystem.

The control unit, together with one or more sound capturing devices suchas microphones, and one or more image capturing devices such as videocameras are placed strategically about the desired location. The desiredlocation can be any place where automobile accidents are likely tooccur, such as busy stretches of road or intersections.

The microphone and video cameras are connected to the control unit sothe control unit can receive and process acoustical data from saidmicrophones and video images from the video cameras. This connection maybe direct, or by wireless means such as a wireless network, Bluetooth,infrared, or any other wireless means of connecting two devices. Inalternate embodiments, the microphones and video cameras may becontained within the housing of the control unit.

In alternate embodiments, a plurality of control units in closeproximity may communicate with each other, for example using a wirelessnetwork or ad hoc network. In cases where the sensor systems of suchcontrol units overlap, the qualifying or preliminary sounds detected atone control unit may be used to commence recording at another controlunit, to thereby increase the available data. A networking of controlunits allows a large sensor network to track events over a broadgeographic region. This network may, for example, be used to track themovements and/or driving patterns of vehicles around an incident, and toidentify and track drivers who leave the scene of an accident.

The microphones and video cameras can be placed anywhere about thedesired location including on or underneath traffic signals, attached toutility poles or other structures such as nearby buildings. The objectis to position one or more microphones and video cameras such as to beable to detect acoustical signals coming from about the desired locationand to provide useful images of an accident at the desired locationincluding the occurrence of the accident itself, pre- and post-accidentimages of the desired location, vehicle identification information,injured parties, and the state of the traffic signal before during andafter the accident.

In the preferred embodiment, if the desired location is an intersectionequipped with traffic control signals, one of the video cameras can bedirected at the traffic signal, or be positioned to cover a portion ofthe traffic signal in order to record and communicate its state before,at the time of, and immediately following an accident. Thisadvantageously bypasses a logical indication of traffic control devicestate, which can in some instances be in error.

In alternate embodiments, in addition to or in place of using videoimages to record the state of the traffic control signal, the controlunit is connected directly to the traffic signal control device by wiredor wireless means, and can record the state of the traffic controlsignal electronically when preliminary or qualifying sounds aredetected.

While microphones and video cameras are the preferred means forcapturing acoustical signals and video images, other sound capturingmeans and image capturing means currently in use or invented in thefuture may be utilized for this purpose.

At intersections or other roadways with existing sensors, such as groundloops or weight sensors, the system may interface to these sensors toprovide additional information.

The control unit also uses a satellite navigation system andcommunication means. In alternate embodiments these may be external tothe control unit and connected to the control unit either directly or bywireless means as with other components of the system.

In the preferred embodiment, the satellite navigation system receiver isa NAVSTAR Global Positioning System (“GPS”) receiver, and is mountedinside the control unit. The GPS receiver is used for determining theexact location and time of an accident.

Using a GPS receiver to determine location and time is highly accurateand enables the invention to be deployed anywhere without the need foradditional programming. This simplifies the deployment process andeliminates the possibility of recording and transmitting an incorrectlocation or erroneous timestamp.

The highly accurate and reliable GPS system is operated by the UnitedStates government and is the preferred means to use with this inventionto determine location and time. However, in alternate embodiments, anysatellite navigation system such as GLONASS or some of the commercialsystems now in the planning stages or to be developed can be utilizedfor the purpose of obtaining location and timing data. In otheralternative embodiments, means other than a satellite navigation systemreceiver can be used for determining time and location including but notlimited to internal time keeping means, programming of the location oridentification information into each individual unit, using land basednavigation signals, or determining of location using one or morecellular or wireless transmission towers.

In the preferred embodiment, the communication means is a wirelesstransceiver housed inside the control unit, and can be any one of thestandard cellular transceiver technologies, including but not limited toanalog cellular (AMPS), Cellular Digital Packet Data (CDPD), Microburst,Cellemetry, digital cellular, PCS GSM, GMRS, GPRS, CDMA, TDMA, FDMA, orany other wireless communication means, including so-called 2.5G and 3Gtechnologies. If necessary, an optional modem is used to convert thesignal from analog into the correct digital format. In alternateembodiments, RF technologies like two-way radio can be used which mayalso require a modem, or the control unit can be connected directly tothe remote monitoring center over the public switching telephone lines(PSTN), or by a wired or wireless network.

In the preferred embodiment, the communication means can also receive anincoming connection from a remote location for the purposes ofdiagnostics and troubleshooting, adjustments to programming, command andcontrol and to reset the unit. For example, if construction is takingplace in the vicinity of the control unit, it can be temporarilydisabled or programmed to ignore those particular construction sounds tominimize the risk of a false alarm. Command and control features canpermit remote adjustment of microphone and camera levels, disabling amalfunctioning microphone or camera, and resetting or disabling of thecontrol unit. Security means can be utilized on the incoming connectionin order to minimize the risk of unauthorized users gaining access tothe control unit programming. Such means for securing electronic devicesare numerous, well known in the art, and need not be discussed furtherhere.

Regardless of how communication from and to the control unit isachieved, the object is to have a means for the control unit to contactthe desired remote location and to transmit the accident related datafor reporting and permanent storage, and to enable command and controlof the control unit from a remote location.

In operation, the control unit continually receives input of acousticaldata from the microphones. This acoustical data is processed in thecontrol unit to determine if the acoustical data received from themicrophones match the acoustical pattern of sounds that indicate a motorvehicle accident is about to occur (“preliminary sounds”) or that amotor vehicle accident is occurring (“qualifying sounds”). For example,the sound of skidding tires is often followed by a collision ofvehicles.

In order to differentiate accident-related sounds from ordinary soundsthat occur at a traffic location, baseline or threshold acousticsignatures of various accident sounds (or models, algorithms, ordescriptions thereof, or matched filters therefor) are stored in thecontrol unit, and all acoustical data received from the microphones aremeasured and compared against these threshold acoustic signatures todetermine if they are ordinary sounds, preliminary sounds or qualifyingsounds. For example, the sounds received may match an acoustic signatureof skidding tires (preliminary sounds) or the acoustic signature of avehicle crashing into another vehicle, or other sounds common at anaccident scene such as a vehicle crashing into an object or hitting apedestrian (qualifying sounds). Any acoustic data received by thecontrol unit with an acoustic level matching the stored threshold levelswill automatically trigger the process of storing accident-related dataaccording to the following parameters. In alternate embodiments, theseparameters may be modified according to the requirements of the agencydeploying the system.

In alternate embodiments, analysis of video images of motor vehiclesmoving through the desired location can be used in place of, or tosupport the use of, acoustic data to detect an accident. For exampleunusual movements like sudden deceleration, acceleration or lateralmovement of one or more vehicles can indicate an accident condition. Aswith acoustic signals, models or algorithms can be used to analyze videoimages for unusual movements, changes in traffic flow or otherindications of an accident.

Generally, the control system will include both models of particulartypes of incidents, as well as a generic algorithm which detectsexceptional circumstances which might indicate a traffic incident orimminent traffic incident. This allows optimum control over common oranticipated circumstances, with adaptivity to handle uncommon or newcircumstances. It is noted that negative models and algorithms may alsobe provided; that is, acoustic signatures or characteristics which areknown to have low or inverse correlation with a type of traffic incidentsought to be detected. For example, it is common to have constructionwork near intersections with steel plates placed over work-in-progress.The sounds of vehicles passing over these plates may be substantial, yetdistinctive. By selectively detecting and filtering these sounds,interference with detection of other sounds, and generation of falsealarms, may be avoided.

One embodiment of the invention provides for on-site calibration andtuning of the control system to account for the environment of use andcontext. This may be especially important for acoustic sensors andprocessing algorithms, although a corresponding tuning process may beperformed with other sensor types. Essentially, the tuning process mayinclude, for example, four different types of standardized acousticpattern excitation. A first type includes impulse noise, such as anexplosion or rapid release of gas, typically useful for a time-domainanalysis of the acoustic environment. A second type includes naturalsounds, generally corresponding to the embedded models, which can begenerated by acoustic transducers or mechanical and generallydestructive means, e.g., breaking glass. A third type includes constantor slowly varying frequency emissions, generally from an electronictransducer, horn or whistle, useful for a frequency domain analysis ofthe acoustic environment. A fourth type includes a pseudorandom noisegenerator, similar to pink noise, generally available only from anelectronic source, to analyze operation in hybrid time-frequency domain.Advantageously, the second (except for destructively generatedemissions), third and fourth types of test equipment may be integratedinto a single unit, capable of producing arbitrary acoustic waveforms.The first type has as its principal advantage the ability to efficientlyproduce high intensity emissions, and therefore will not generally be anelectronically produced emission. By providing an as-implemented activetuning of the system, it is possible to shorten the training time foradaptive features of the control, while simplifying the algorithms, ascompared to a control system which is deployed without any specifictuning process. Likewise, updating of the algorithms and acousticsignatures is also simplified, since the tuning data may be maintainedseparate and therefore applied to an updated model.

In order to reduce the burden on the agency deploying the system, it ispreferred that the control unit be deployed in a generic manner and thenautotune itself for acoustic conditions at the desired location. Forexample, as a part of the installation process, various sounds may besimulated or generated, allowing the control unit to calibrate itselfunder known conditions. For example, an audio transducer may be placedat an appropriate location to generate acoustic patterns associated withvarious traffic incidents. A technician may intentionally break a testpiece of glass, or otherwise generate actual sounds of a characterexpected during a traffic incident. Impulse noises, such as a smallexplosion, gunshot (preferably a blank), balloon pop, or other intenseand short sounds may be generated to help map the acoustic environment.Likewise, extended sample sounds, such as air or steam horns, acoustictransducers generating single frequencies, multiple frequencies, whitenoise, etc., may also be used to map the acoustic environment. During aperiod after initial installation, the system may be remotely monitored,e.g., continuously, to analyze ambient sounds and ensure that thevarious sensors are operating and the thresholds are set appropriately.

It is therefore an aspect of one embodiment of the invention that acustomized sensor system is obtained through installation of arelatively standard set of hardware, with a minimum of on-site work. Itis a further aspect of one embodiment of the invention that aninstallation (and optionally maintenance) procedure is performedincluding an analysis of the acoustic environment and context, to ensureadequate system operation with standardized hardware and software, andto permit optimization on-site.

In the preferred embodiment, the control unit is continually storing inthe buffer (RAM or data storage means), all audio signals and videoimages of the desired location in a circular buffer or loop that goes onfor a specified period of time, overwriting audio and video that exceedsthe specified period of time. When a qualifying sound is detected, thecontrol unit stops overwriting and saves the stored audio signals andvideo images leading up to the qualifying sound. The time and locationdata at the time of detection of the qualifying sound are recorded ifdesired, and if the control unit is connected to a traffic signalcontrol unit, the state of the traffic control signals at the time ofdetection of the qualifying sound can also be recorded. Subsequent tothe qualifying sounds, the control unit continues saving audio signalsand video images until the accident is reported, the accident relateddata is transferred to a remote location and the control unit is reset.If desired, the saving of audio and video data can be stopped after apredetermined amount of recording time passes, or upon command by theoperator from a remote location. In this embodiment, the system is notdependent on preliminary sounds, and is designed to capture the eventsleading up to an accident. This can be particularly useful indetermining the events leading up to the accident, the cause of theaccident, assessing fault and determining liability.

In an alternate embodiment, both preliminary sounds and qualifyingsounds are utilized, making it unnecessary to continually record audiosignals and video data prior to the occurrence of a preliminary sound,as the recording starts upon either of detecting a preliminary orqualifying sound.

In such alternate embodiments, when the control unit detects apreliminary sound like the sound of skidding tires, the control unitbegins storing all subsequent audio data and video images. At thispoint, the time and location data at the time of detection can berecorded if desired, and if the control unit is connected to a trafficsignal control unit, the state of the traffic control signals at thetime of detection of the preliminary sound can also be recorded.Activating the recording process based on preliminary sounds enables therecording of audio data and video images of an accident to start in themoments before the accident occurs and does not require the storing ofaudio and video data prior to a preliminary or qualifying sound. If apreliminary sound triggers recording, the location, time and state ofthe traffic signal can be recorded again upon the detection of aqualifying sound.

If a pre-determined amount of time elapses after a preliminary sound andno qualifying sound is detected, meaning that a potential accident didnot occur, the control unit stops recording audio data and video images,the recorded data is cleared from the system, and the control unitresumes its normal operation monitoring for preliminary or qualifyingsounds.

Regardless of the embodiment, when the control unit detects a qualifyingsound, meaning that an accident is occurring, storing of audio data andvideo images continues for a predetermined length of time (or startsimmediately if there was no preceding preliminary sound in alternateembodiments that utilize preliminary sounds), location and time data arerecorded by the control unit, and if connected to a traffic signalcontrol unit the state of the traffic control signals at the time ofdetection of the qualifying sound is also recorded.

There are sometimes instances when an accident can occur without anyadvance warning including the absence of preliminary sounds. In thepreferred embodiment, the audio signals and video images leading up tothe qualifying should have been saved regardless of the circumstancesleading up to the qualifying sounds. In alternate embodiments thatutilize preliminary sounds, if a qualifying sound is detected withoutany preceding preliminary sounds, such as an accident where neitherdriver has the opportunity to apply the breaks prior to impact, theentire process described above, including the storing of audio data andvideo images, begins immediately upon detection of the qualifying sound.

Regardless of the embodiment, when a qualifying sound is detected, thewireless transceiver begins to initiate contact with the designatedremote location (“the monitoring center”). The control unit willcontinue attempting to establish contact with the monitoring centeruntil contact is established. The system may provide a time-out whichceases communications attempts after a predetermined amount of timelapses, to avoid undue communication system burden in the event of afailure. If communication is not immediately established, there are anumber of options available. To the extent possible, the remote unit maystore data internally until communications are established. The remoteunit may also employ a redundant or backup communications link, forexample an alternate cellular carrier, ad hoc network, satellitecommunications, or other secondary communications system. In the eventthat the impairment is not with the communications channel, but with themonitoring center, the data may be sent to an alternate or backupmonitoring center.

The monitoring center can be an alarm company that monitors commercialand residential alarm systems, many of which have been around for years,a vehicle monitoring service many of which have started operations inthe recent years since auto manufacturers have started equippingvehicles with GPS receivers, a monitoring center establishedspecifically for the purpose of the monitoring roadways equipped withthe instant invention, or the dispatch center for local fire, police andemergency. Typically at these facilities, an operator at a workstationwill see images of the accident scene and location data on a videomonitor. Prompts can be provided to instruct the operator steps to takewhen an accident is reported, including giving the contact informationfor the emergency response agency in that location. Such systems foroperating a monitoring center as described are well known in the art andneed not be discussed further here.

Known and existing systems and services may readily lend themselves foruse with the instant invention, provide a more economical solution forthe agency deploying the system, and can use excess capacity and provideadditional revenue opportunities for the operators of these services,although it may be desirable to provide operators as such facilitieswith specialized training. However, there are instances whereestablishing an independent service or integrating the service intoexisting dispatching facilities of the local authorities might be thepreferred solution.

In the preferred embodiment, when the transceiver has established aconnection with the remote location (“the Monitoring Center”), thecontrol unit initially transmits the location and at least one stillimage or live video image of the accident scene from at least one of thevideo cameras. The determination of whether to use a single or multiplestill or moving images at this step is preprogrammed into the controlunit according to predetermined settings as desired by the agencydeploying the system. Other accident-related data can also be sent withthe initial contact, also depending on pre-programmed preferences. Theamount and type of data transmitted upon initial contact will bedetermined in part by the communication means being used, the connectionspeed and available bandwidth, but the object of the invention is toquickly and efficiently notify the monitoring center of the location ofthe accident and provide the operator with at least one still or movingimage of the accident scene to allow the operator to access the accidentscene.

The location data and video images of the accident scene beingtransmitted from the control unit are displayed on a video monitor atthe monitoring center where a person (“the operator”) can assess thelocation and severity of the accident, notify the proper authorities,and provide useful information to help the authorities determine anddispatch the appropriate level of emergency response. If the monitoringcenter is being operated by the local authorities, the emergencyresponse can be dispatched directly by the operator

After the authorities have been notified, the operator at the remotemonitoring center can initiate a transfer of the accident-related datastored at the control unit to permanent storage at the monitoring centeror other designated facility, or this process can be programmed to takeplace automatically without operator intervention thereby minimizing therisk of losing accident related data due to human error. Thetransmission of stored accident-related data can also start and continueto take place while recording continues and the authorities are beingnotified.

Error checking methods known in the art or to be developed can beutilized to make certain that the accident related data is correctly andcompletely transmitted and stored in a permanent record at themonitoring center or desired location. Such error checking methods arewell known in the art and need not be discussed further here.

When the accident-related data has been successfully stored in apermanent record, the control unit may be programmed to unprotect thepersistent data storage system, allowing subsequent events to be stored.If the connection with the central monitoring center is kept open, thismay be closed, and the system may resume normal operating status,waiting for the next preliminary or qualifying sound to occur. Thisprocess can occur automatically, or can require a deliberate signal besent from the monitoring center.

Typically, it is preferred that the location monitoring units berelatively autonomous, as well as fail safe, and therefore preferably donot require significant handshaking or dense communications in order tomaintain normal operating conditions. Therefore, it is preferred thatthe location monitoring units continuously operate to track conditionsor events at the location, regardless of operational conditions at thecentral monitoring center, and regardless of any communicationsimpairments which might occur.

Once the accident-related data is received from the control unit andsaved to a permanent record, this permanent record can then be madeavailable to the authorities for use in determining the cause and faultfor the accident, and can be used by the courts, insurance companies andthe victims in determining and settling fault and liability.

It is therefore an object of the invention to provide an automobileaccident detection, reporting and recording system that uses sound, orother non-visual cues, to determine if a vehicular accident hasoccurred, or is about to occur, and if so, to maintain a record ofaccident-related sounds and images, together with other data such astime, location and state of the traffic signals, for a period of timeprior to or at the time the accident is detected, and for a period oftime thereafter. The record is then reported to a central repository,both for archival storage and to enable a person at such facility toassess the severity of the accident and dispatch an appropriateresponse. It is noted that the emergency control response center neednot be the same as, or related to, the archival storage center, andindeed these can be independently owned, controlled, and operated.Likewise, the economic models underlying these functions can beindependent. In fact, it would be reasonable for those at fault in anaccident to be assessed a fee for the emergency response expenses, aswell as to pay for fees for using the monitoring system andinfrastructure. This could be considered a tax, fine, or user fee.

It is a further object of the invention to provide a system formonitoring a location, comprising, an audio transducer for detectingacoustic waves at the location, and having an audio output; a processorfor determining a likely occurrence of a vehicular incident, based atleast upon the audio output; an imaging system for capturing videoimages of the location, and having an image output; a buffer, receivingthe image output, and storing a portion of the video images for apreceding period, including at least a period immediately prior to thedetermination of a likely occurrence of the vehicular incident up to atime of the determination; and a communication link, for selectivelycommunicating the portion of the video images stored in the buffer,wherein the buffer retains the portion of the video images, at leastuntil an acknowledgement of receipt is received representing successfultransmission through the communication link, and after receipt of theacknowledgement, a portion of the buffer containing the portion of thevideo images is available for reuse.

It is also an object of the present invention to provide a modularintelligent transportation system, comprising an environmentallyprotected enclosure, suitable for protecting electronic equipment fromuncontrolled weather conditions when placed outdoors, the enclosurebeing installed at a location, a bus, providing a communicationsinterface between at least two devices connected to the bus, a port forat least one of receiving a signal indicating a state of a trafficsignal, and producing a signal for controlling a state of a trafficsignal, a processor module and memory, communicating with the bus,having at least one of an audio data input, and an image data input, theprocessor module analyzing data patterns represented in at least one ofthe audio input and image data input and audio input, and the memory atleast temporarily storing at least a portion of the data, at least oneoption port, providing an available location for hardware and physicalaccess to the bus, wherein the processor module has sufficientadditional processing capacity to undertake additional processing tasks,a power supply, providing power to the processor module and to an optionport or device connected to said option port; and a communication linkfor communicating remotely from the enclosure.

It is a further object of the invention to provide a vehicularmonitoring system, comprising an environmentally protected enclosure,suitable for protecting electronic equipment from uncontrolled weatherconditions when placed outdoors, a power supply, a processor module forreceiving data representing vehicular conditions and analyzing the datato determine at least one of a vehicular incident, a likely occurrenceof a vehicular incident, or an imminent vehicular incident, and memoryto at least temporarily store at least a portion of the data, aninterface for communicating with a traffic signal control device,receiving at least a status of the traffic signal, at least one optionport, providing a physical interface for communications with theprocessor module, and a communication link for communicating remotelyfrom the enclosure, wherein the at least one option port permitsoptional hardware therein to access resources of at least one of thepower supply and the processor module.

It is another object of the invention to provide a modular intelligentroadway monitoring system, comprising an enclosure, providingenvironmental protection to electronic modules contained therein, acommunication link for communicating data remotely from the enclosure, avehicular monitoring system, receiving an image input and communicatinginformation relating to the image input remotely through thecommunications link, and an interface for communicating with an optionalmodule proximate to the enclosure, and providing logical communicationswith the vehicular monitoring system and the communication link.

The system may further include a wireless data communication system, forexample a cellular communications transceiver, a multihop ad hoc networkradio transceiver, or at least one wireless local area network accesspoint, having at least one of a communications path to thecommunications link and the at least one port. The communication linkmay provide unidirectional or bidirectional communications remotely fromthe enclosure. The port may be capable of receiving information relatingto local vehicular conditions proximate to the enclosure. Thecommunications link may be a packet switched network, for example, theInternet or other TCP/IP network. The communication link may include aprimary link and a backup link, using a different physical transportlayer, the communication preferentially occurring through the primarylink, and in an event of failure of the communication through theprimary link, then through the backup link, wherein a physical transportlayer of the primary link and the backup link are each selected from oneor more of the group consisting of public switched telephone network,coaxial cable, twisted pair, cellular communications, satellite-basedcommunication system, point-to-point radio frequency wireless,point-to-point microwave wireless, broadcast wireless, line-of-sightoptical, fiber optic, and ad hoc radio network. Data identifying thelocation may also be communicated over the communications link. Thevarious communications systems may intercommunicate and relay messagesthrough each other. Therefore, a wireless local area network accesspoint may provide Internet access around the system enclosure throughthe communication link, which is also used for core system functions.This same access point may be used to communicate with various sensors,such as camera and microphones, and with a traffic signal controldevice.

The processor module may execute an operating system having anapplication programming interface such that hardware placed in theoption port can communicate with the processor module and interact withthe application programming interface. The system may provide forensicreliability for at least one type of data stored by the processor moduleand communicated through the communication link. The processor modulemay determine a likely occurrence of a vehicular incident, based atleast upon the audio input. The processor module may also persistentlystore a portion of the image data for a period preceding adetermination, wherein the portion of the image data may be preserved atleast until an acknowledgement of receipt is received representingsuccessful transmission through the communication link. The processormodule may determine the likely incidence of a vehicle incident at thelocation based on occurrence of at least one of a set of predeterminedaccident related acoustic signatures represented in the audio input. Theprocessor module may analyze a motion vector of at least one vehicle atthe location to determine the likely occurrence of a vehicular incidentand/or correlate a pattern of the motion vector with an acousticinformation pattern to determine the likely occurrence of a vehicularincident. Another possible function of the processor is to determinewhether a vehicle fails to comply with an applicable traffic signal,traffic sign, law, rule, or regulation. The processor may also predictand/or detect various types of vehicle incidents.

The system may include a satellite based geolocation system, or aterrestrial based geolocation system for determining a geographiclocation of the system. The system may also contain a self-containedpower source, which may be a rechargeable battery system primary batterysystem, solar or wind power system, internal combustion engine withgenerator, or the like. The system preferably includes a camera forinputting image information and/or a microphone for inputting acousticinformation relating to vehicular conditions.

The system architecture may include a remote monitoring center at theremote location, communicating with the system through the communicationlink, the remote monitoring center receiving communications from aplurality of communications links at respectively different locations. Arecord communicated through the communications link, preferably one thatmay be forensically authenticated through a manual or automated process,providing at least one of physical security and cryptographic security,to ensure that the forensic record is a reliable record of eventsrecorded by the system. Although a forensic record is preferred, therecord can be communicated normally, without any forensic or securityprotocol.

The system may include an interface to a traffic signal control device,although a camera may also be used for the purpose of determining thestatus of any traffic signals at the location. The system may alsoinclude a power supply for providing power to the vehicular monitoringsystem and an optional module, which for example is contained within thesame housing as the system processor. The optional module may interfacewith a communications bus through either a modular connector or awireless connection. The optional module may also be located outside thehousing, the optional module and the system processor communicatingthrough a data communications network, wherein communications betweenthe optional module and an external system are conveyed through thevehicular monitoring system.

The communication link may comprise a wireless transceiver, whichgenerally simplifies installation. Alternately, the communicationsphysical transport layer can include coaxial cable, twisted pair,cellular communications, point-to-point radio frequency wireless,point-to-point microwave wireless, line-of-sight optical, fiber optic,and ad hoc radio frequency network. According to one embodiment, thecommunication link comprises a primary link and a backup link, using adifferent physical transport layer, the selective communicationpreferentially occurring through the primary link, and in an event offailure of the selective communication through the primary link, thenthrough the backup link. The backup link, in this case, may employ amore expensive communications method. This, in turn, allows selection ofa less expensive physical transport layer for the primary link, even ifthe reliability of this is less than required.

The system may further comprise a location sensor, for determining ageographic position of the location, the geographic position beingcommunicated through the communications link. The location sensor is,for example, a GPS receiver, receiving signals from a plurality ofcommunication satellites and determining a geographic position of thelocation and a time, the geographic position and time being communicatedthrough the communications link. Therefore, for example, thecommunication link is wireless, and the system communicates informationdefining its location to a remote system. The location information isuseful since a plurality of systems may employ a common wirelesscommunications band, and thus cannot be distinguished based on aphysical communications channel employed.

The buffer may receive at least one of the audio output and videooutput, and store a portion of the audio output representing theacoustic waves and video output representing video images for apreceding period, including at least a period immediately prior to thedetermination of a likely occurrence of the vehicular incident up to atime of the determination, wherein the communication link selectivelycommunicates the portion of the audio output stored in the buffer. Thecommunication link may also communicate a stream of video imagescaptured after the determination. The audio transducer comprises, forexample, one or more microphones.

The processor may formulate its determination based on occurrence of atleast one of a set of predetermined accident related acoustic signaturesrepresented in the audio output. The processor may determine a likelyimminence of a vehicular incident, based at least upon the output of theaudio transducer and the immediately preceding period extends between adetermination of a likely imminence of a vehicular incident and a likelyoccurrence of a vehicular incident. Alternately or in addition, theprocessor may analyze the image output to determine a likely imminenceand/or occurrence of a vehicular incident.

The system may also include a self-contained power source to operate thesystem in the event of a power failure.

The communication link typically communicates with a remote monitoringcenter, the remote monitoring center generating the acknowledgement ofreceipt. The communications link may also communicate with mobiletransceivers, for example to supply information to a vehicle regardingconditions at the location. Thus, the system may form part of areal-time telematics system to provide traffic information to vehicleson demand. The remote monitoring center may issue control signals to thesystem through the communications link, thus allowing remote control ofthe system, for example, even when no incident is detected.

The imaging system may comprise a plurality of video cameras directed atvarious portions of a location near an electrical traffic signal,wherein a first video camera is activated for a predetermined timeperiod and each subsequent video camera is activated upon deactivationof an active video camera such that only one the video camera isactivated at a given time. This configuration permits the system tooperate with a limited set of resources, for example a singlemultiplexed video input. The imaging system may also comprise aplurality of video cameras directed at various portions of a location,in which the processor produces a combined output representing asynthetic representation of the location. A synthetic representation istypically more useful for real time streaming of data to provide highcompression ratios of data representing a single set of objects frommultiple sensors, rather than forensic evidence, since the synthesis maybe prone to certain types of errors. The communication link may beoperative to activate the system to communicate video images based on aremote request.

The system may also include a traffic control device status sensor, thetraffic control device status being communicated by the communicationlink.

It is a further object of the invention to provide a system fordetermining and reporting the occurrence of a vehicle incident at ascene comprising a sensor for detecting conditions at the scene; meansfor predicting the likely occurrence of a vehicle incident at the scene,based on a comparison of detected conditions from the sensor and a setof predetermined incident signatures, the means for predicting producingan output prior to or contemporaneous with the vehicle incident; amemory for storing conditions at the scene detected by the sensor; and acommunications system for selectively communicating the storedconditions to a remote monitoring center after predicting a likelyoccurrence of an incident, including conditions detected preceding thelikely occurrence of a vehicle incident.

The sensor may comprise one or more microphones and/or video camerasadapted to capture incident-related audio or video signals at the scene.Further, sensors may also include radar transceivers, and LIDARtransceivers.

The memory may comprise a circular buffer, wherein contents of thecircular buffer are preserved after a prediction of a likely occurrenceof an incident until an acknowledgement is received that the contentshas been communicated to a remote location.

The system may also comprise a location sensor, for determining alocation of the scene, the location being communicated through thecommunication system.

In accordance with an embodiment of the invention, the system may have alow resource mode and a high resource mode, the low resource mode beingactive prior to a prediction of a likely occurrence of an incident, thehigh resource mode being active subsequent to a prediction of a likelyoccurrence of an incident until reset, wherein the system has a limitedcapability for maintaining the high resource mode. For example, theresource limitation may be availability of power or memory capacity.

It is a still further object of the invention to provide a methodcomprising the steps of capturing vehicle incident-related signals at ascene; determining if a vehicle incident has occurred at the scene;capturing incident-related data preceding and during the occurrence ofthe determined vehicle incident; transmitting the capturedincident-related data; and protecting the incident-related data until anacknowledgement is received indicating successful receipt of theincident-related data by a remote system, then unprotecting theincident-related data, wherein protected incident-related data isselectively preserved. The determining step may comprise analyzing anaudio signal for acoustic emissions which have a high correlation withan incident, and/or analyzing a video signal to determine object statesand vectors which have a high correlation with an incident. A compresseddigital signal may be transmitted representing a composite of aplurality of sensor signals representing conditions at the scene. Astream of real time video data representing conditions at the scene mayalso be transmitted.

In accordance with these and other objects which will become apparenthereinafter, the instant invention will now be described in itspreferred embodiment with particular reference to the accompanyingdrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a perspective view of a typical traffic intersection sceneincluding a preferred embodiment of the automobile accident detectionand data recordation system of the present invention;

FIG. 2 is a perspective view of a typical traffic intersection sceneincluding an alternate embodiment of the automobile accident detectionand data recordation system of the present invention;

FIG. 3 is a flowchart representing the steps performed by the automobileaccident detection, data recordation and reporting system according to afirst embodiment of the present invention;

FIG. 4 is a flowchart representing the steps performed by the automobileaccident detection, data recordation and reporting system according to asecond embodiment of the present invention;

FIG. 5 is a block diagram of a system according to another embodimentthe present invention; and

FIG. 6 is a flowchart representing steps of a method according to theembodiment of FIG. 5.

FIG. 7 shows a schematic view of an MITS, showing a single boardcomputer main module and open slots for optional modules, as well asvarious communications facilities.

FIG. 8 shows a schematic view of an MITS, showing an adjacent accessorycomponent.

FIG. 9 shows a schematic view of an MITS, showing an interconnectedaccessory component.

FIG. 10 shows external connectivity connections and various options forthe MITS.

FIGS. 11A and 11B show aspects of an alternate embodiment of theinvention employing a video-enabled cellular telephone handset at aremote location, showing the camera and microphone connected to a MITS,and the MITS and associated sensors at an intersection, respectively.

FIG. 11C shows an embodiment of the invention which employs a cell phonehaving GPS, a cellular transceiver, a WiFi and/or WiMax radio, a camera,a microphone, an expansion slot, and a data port.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

As seen in FIG. 1 the present invention is illustrated and generallydesignated as the system 100. The system 100 comprises one or morelistening devices 15 placed proximate a traffic scene 1 which isreferred to as the desired location. The desired location 1 can be anystreet, a section of highway, an intersection, or any other place wherea traffic accident can occur. Listening devices 15, preferablymicrophones, are be mounted strategically at one or more positionsproximate the desired location 1. In FIG. 1, the microphones 15 areplace on utility poles 20, but they can be placed on any objectproximate the desired location 1 such as underneath the traffic signals30, suspended on wires above the intersection as shown in FIG. 2, or onother structures such as buildings so long as they are placed to allowaccurate capture of the acoustic signals at the desired location 1.

The microphones 15 are connected to the MITS, also referred to as thecontrol unit 25 either by wired or wireless means, and the control unit25 receives the acoustic signals from the microphones 15 and convertsthem to a data format that can be compared to the acoustic signatures ofaccident related sounds. These accident related sound signatures caninclude the sound of skidding or screeching tires (preliminary sounds)or the sound of a vehicle impacting another vehicle, structure orpedestrian (qualifying sounds), all of which indicate an accident isabout to occur or is occurring. Further, the acoustic signals receivedfrom the microphones 15 can be filtered to remove sounds which aregenerally non-indicative of traffic incidents or accidents. This furtherinsures that the control unit 25 will detect and react only to soundsthat have a high probability of being accident-related sounds.

It is also possible to use a passive (non-electronic) acoustic pickupdevice. For example, a laser beam incident on a diaphragm will bemodulated by the acoustic vibrations present. Likewise, passive radiofrequency devices (e.g., backscatter emitting) devices may be sensitiveto acoustic waves. Therefore, the control unit 25 may emit energy whichis modulated by the acoustic waves in the environment, which is thendetected and used to determine the audio patterns.

In this preferred embodiment the control unit 25 needs only to react toaudio signals determined to be qualifying sounds, such as the sounds ofan actual impact of a vehicle with another vehicle, object orpedestrian, because data is continually saved in a circular buffer, andupon occurrence of a qualifying sound the buffer temporarily stopsoverwriting old data, or transfers the data from a temporary buffer topersistent storage, thereby preserving a record of the accident. Thispreferred embodiment can, but does not need to, respond to preliminarysounds.

In alternate embodiments, the system also reacts to audio signalsdetermined to be preliminary sounds indicating an accident is about tooccur such as the skidding of automobile tires, and starts recordingdata when it detects either a preliminary or qualifying sound. Thisalternate embodiment can, but does not necessitate, the use of acircular buffer.

The circuitry for determining whether the received acoustic signals arequalifying sounds (or preliminary sounds in alternate embodiments) ishoused within the control unit 25 which also houses some othercomponents of the system 100. FIG. 1 shows control unit 25 mounted on autility pole 20 although the control unit 25 can be situated upon anystructure proximate the desired location.

Typically, this circuitry will include a digital signal processor,although a microprocessor may be programmed to perform digital signalprocessing with its general purpose computing resources.

To accurately capture images related to the accident, it is necessary toplace one or more image capturing devices, preferably video cameras 35,at such positions that they can capture video images of the desiredlocation 1. The video cameras 35 can also be used to determine thestatus of traffic signals 30, and if so desired one or more videocameras 35 may be directed at the traffic signals 30. Ideally, the viewangle of the video cameras is sufficiently wide to display both thestreet area of the desired location 1 and the visible portion(s) of thetraffic signal(s) 30 from that angle, however, a separate video cameraor cameras 35 or other suitable devices can be used exclusively tomonitor the state of the traffic signals at the desired location 1.Alternatively, the control unit 25 can be connected to the trafficsignal control device 36 in place of or in addition to the use of videocameras 35 for this purpose.

The video cameras 35 are positioned proximate the desired location 1,preferably on utility poles 20 as shown in FIG. 1, or on otherstructures at or near the desired location. In one configuration, thecameras are suspended above the center of an intersection as shown inFIG. 2. It is preferred, as shown in both FIGS. 1 and 2, that fourcameras be situated such that images of all possible areas near thedesired location 1 are captured, and each camera 35 is electrically orwirelessly connected to control unit 25 using means similar to the meansused to connect the microphones to the control unit 25. However, it maybe desirable to use more or less than four cameras 35. For example, onecamera 35 may be mounted in a location such as a building with a viewthat covers the entirety of the desired location 1 and at least one ofthe traffic signals 30.

In the preferred embodiment, the video cameras 35 are always activatedand always sending video images to the control unit 25. The control unit25 continually saves audio signals and video images to a circular bufferin a loop for a predetermined period of time, overwriting audio andvideo data that falls outside this predetermined time range. This schemetherefore allows persistent storage of prior events, while minimizingmemory usage and preserving privacy of persons near the incident attimes when there is no incident.

In alternate embodiments, storing of audio signals and video images istriggered only by the detection of a preliminary sound or by aqualifying sound if there has been no preceding preliminary sound.

In yet another alternate embodiment the cameras 35 are in the off orstand-by condition, and when a preliminary or qualifying sound isdetected at the desired location 1, the control unit 25 sends a signalto each camera 35, activating them so recording of images can begin. Inother alternate embodiments, a series of cameras 35 may be programmedfor each to be active for a predetermined length of time, so that imagesfrom at least one video camera 35 is always available for capture shouldan accident occur. The cameras 35 may be associated with motiondetectors, or themselves be used as motion detectors, to trigger videotransmission and recording. For example, a first camera 35 may beoperating from time T₁ until time T₂, at which time it shuts off. Justprior to T₂, a second camera 35 is activated and begins recording imagesat the scene until time T₃. Just prior to time T₃ a third camera 35begins operating. This sequence can continue for additional cameras 35,reverting back to the first camera 35 again. This allows for continuousmonitoring of the desired location 1 by a select number of video cameras35 while optimizing the cameras' 35 available resources until they areneeded. The timing and operation of each camera 35 is controlled fromcontrol unit 25. In this alternate embodiment, when the control unit 25detects a preliminary or qualifying sound, all cameras can becomeactive, but the control unit 25 is able to capture the image from thecamera 35 that was active at the time of the qualifying or preliminarysound without any lag time that may occur while the other cameras 35take time to turn on or activate. Alternatively, one or more specifiedcameras 35 can be left on all the time, and others activated upondetection of a preliminary or qualifying sound. Discussion of thesealternate embodiments, here and throughout this description is notintended to be limiting, and the intent is to illustrate some of themany possible combinations for configuring and customizing the system100.

By limiting required data flows between the elements based onintelligent analysis of the data or the use of heuristics, greaterefficiency is obtained, permitting deployment of a design having lowercost, and causing less interference or intrusion into its environment orcontext. Thus, while all data may be continuously recorded andtransmitted, this is relatively inefficient and intrusive.

Reference is also made to the components in FIGS. 1 and 2. In thepreferred embodiment, the control unit 25 continually receives andmonitors the incoming acoustic data received from the microphones 15 andanalyzes the acoustic data to determine it corresponds to a pattern of aqualifying sound, for example, the sound pattern resulting from a motorvehicle impacting with another motor vehicle, a pedestrian or an object.In one alternate embodiment, when a qualifying sound pattern isdetected, the control unit 25 may communicate with other nearby controlunits, instructing them to also capture and transmit data. This, forexample, might allow capture of the path of a hit-and-run accidentbefore and after the accident, video from other angles, and the identityof witnesses (through license plate tracking).

In the preferred embodiment, the video camera(s) 35 are always in an“on” state so the control unit 25 is always receiving the video images,and the control unit 25 is always recording audio signals and videoimages in a circular buffer or loop that goes on for a predeterminedperiod of time, continually overwriting data that exceeds thepredetermined period of time. This and other predetermined periods oftime discussed throughout this description, are variables which can beset according to the preferences of the agency deploying the system 100,and indeed, the predetermined period can change in each instance. When aqualifying sound is detected, the control unit 25 persistently storesthe audio and video data that was buffered prior to the qualifyingsound, and begins a sequence of events as described below.

In alternate embodiments that utilize preliminary sounds, if an incomingsound is recognized to be a preliminary sound, then protected storage ofthe audio signals and video images begins and the control unit 25continues to monitor incoming audio signals until the earlier of apredetermined period of time elapses or an incoming audio signal isrecognized to be a qualifying sound.

If before the passing of a predetermined time, an incoming sound isrecognized to be a qualifying sound, meaning a determination that anaccident is occurring, then recording of audio and video signalscontinues and a number of other events are triggered as described below.

If a preliminary sound has been detected and the predetermined timepasses without the detection of a qualifying sound, meaning that anaccident related sound has not been detected, the recording ends, thestored data is cleared, and the control unit 25 returns to “listeningmode” to wait for the next preliminary or qualifying sound.

If an incoming sound is initially recognized to be a qualifying sound,then the storage of audio and video signals begins immediately as itdoes with the detection of a preliminary sound, and the control unit 25proceeds with the other steps described below in the same manner as whena qualifying sound follows a preliminary sound.

It is noted that the hardware which is part of the control unit 25 maybe used for other purposes, such as traffic violation monitoring(compliance with traffic control devices, speed control, etc.).

Returning to a consideration of the preferred embodiment, when thecontrol unit 25 detects a qualifying sound that indicates an accident isoccurring, the control unit 25 initiates the following series of events:

The circular buffer temporarily stops overwriting data, and video datarecorded prior to the qualifying sound, and audio data if desired, issaved and will no longer be overwritten or erased, and all ensuing videoimages, and audio signals if desired, are also stored within a storagedevice which can be RAM memory, a hard drive, magnetic or optical tape,recordable CD, recordable DVD, flash memory or other electronic storagemedia. The storage device can be located within the control unit 25, orin some alternate embodiments can be a separate device connected to thecontrol unit 25 by wired or wireless means. The recording of audio andvideo signals continues for a predetermined length of time. Therefore,the control unit 25 captures events leading up to, during and after theaccident or event occurs.

In addition to recording of video and audio data, a qualifying soundalso triggers the following events:

In the preferred embodiment, a satellite navigation system receiver suchas the Naystar GPS 40, is the preferred means used to determine the timeand location. The time and location may also be determined using othertypes of satellite-based geolocation, such as differential globalpositioning system device (DGPS), GLONASS, Galileo, Argos, andCospas-Sarsat, or a terrestrial network based positioning device, suchas LORAN, cellular network geolocation, or other types of systems, whichmay employ one or more of angle of arrival and/or antenna spatialradiation patterns, time difference of arrival, signal path propagationpatterns, and the like. Alternatively, a location identifier can bemaintained in the control unit 25. Time may also be maintainedinternally within the control unit or determined at the remotemonitoring center 45. For example, the location of the control unit 25may also be programmed or hard-coded into the control unit 25, or alocation identifier may be programmed into the control unit 25 to betransmitted to the monitoring center 45 where the location can be lookedup in a database. While use of pre-programmed location or locationidentifier is functional, it is not the preferred means for identifyinglocation because it is prone to human error and adds to the complexityof deployment, unlike the geo-location means discussed above. In thepreferred embodiment, a GPS receiver preferably located within controlunit 25 constantly receives signals from GPS satellites 40. Upon thedetection of a qualifying sound, the time of detection of the qualifyingsound is determined. While the location is also available from the GPSreceiver, a stationary control unit will typically not need to acquirelocation information for each event, there is little cost in doing so.The GPS data (including, for example a full timecode which specifiestime and date, as well as location) is therefore recorded, stored andtransmitted to the remote monitoring center 45 along with the video dataand optional audio and traffic signal data. Although in some alternateembodiments, the control unit 25 can continue to record the time atspecified intervals and for a predetermined period of time, in thepreferred embodiment the location and time are recorded at least at thetime when a qualifying sound is detected, and either may be recordedwith each image, and if desired and present upon each change in thestate of a traffic control signal(s) 30. In alternate embodiments thatuse preliminary sounds, the time of the detection of a preliminary soundcan also be recorded.

Using the elements described above, a data file or multiple data filescontaining accident-related information such as audio signals, videoimages and GPS time and positioning data, and data on the state of anytraffic signal present at the desired location 1 proximate to the timean incident is detected, is created and stored in memory or other meansas described above. It should be noted that the agency deploying thesystem 100 can select to capture and transmit part or all of theavailable accident-related data according to its preferences, but thatgenerally, at a minimum, the system needs capture and transmit video andlocation data in order to be useful for its intended purpose.

While, in theory, the accident-related information could also be storedlocally, this information has potential forensic value, and this localstorage might necessitate impounding of the control unit 25 as evidence,leading to substantial inefficiencies. On the other hand, if theaccident-related data is reliably and securely communicated to a remotesite and flushed from the control unit 25 as a matter of course, then itis less likely that a forensic analysis will require more than aninspection of the control unit 25, while avoiding impairment of thedata.

Once commenced, the recording and storing of all accident-related datacontinues for a pre-determined length of time, until memory/storagecapacity is reached, or until the data is communicated to a centralmonitoring system (and preferably acknowledgement received). Forexample, the recording process can continue for a minute, severalminutes or fifteen minutes or more, and can be programmed or adjustedremotely from the monitoring center 45 if there is a need to shorten orextend the time of recording.

Returning back to the preferred embodiment, upon the detection of aqualifying sound indicating that an accident is occurring, the controlunit 25 starts to initiate contact with the designated monitoring center45 over the communication link 50. The monitoring center 45 can beoperated by the authorities or agency deploying the system, can be aspecial facility dedicated exclusively to monitoring traffic accidentsor incidents, equipped with the present invention, or, alternatively,can be a standard monitoring center used to monitor incoming alarm callsor transmissions from vehicle navigation systems.

The preferred means of communication link 50 is a wireless system, andany of a number of traditional wireless communication technologies canbe utilized such as cellular, PCS, CDPD (Cellular Digital Package Data),2.5G cellular, 3G cellular, or a data transmission technology developedfor use on cellular phone frequencies; however, contact can beestablished by standard or wireless telephone line or network connectionas well.

Upon making contact with the monitoring center 45, the control unit 25initially transmits the location information of the desired location 1which may be displayed on a computerized map at the monitoring center45. In the preferred embodiment, simultaneously or shortly before orafter the location data is transmitted, at least a still or live imageof the desired location 1 showing the accident scene is transmitted tothe monitoring center 45 and at least the location of the accident isdisplayed, preferably on an electronic map together with at least oneimage of the desired location 1 so the operator at the monitoring center45 can evaluate the accident scene to determine the appropriate level ofresponse. Alternatively, a series of images can be transmitted atpredetermined intervals, or real-time live video can be utilized. Astill image can be used when bandwidth is limited, and a series of stillimages or a live image can be used when sufficient bandwidth isavailable. A still image followed by a live image can be also used sothat the location and image of the accident can be quickly transmittedfor visual assessment by the operator in determining an appropriateresponse, followed by periodic still or live images to allow theoperator to continue to monitor the situation and report to theauthorities. If desired, it is possible to transmit still images havinghigher resolution than that present in the video feed, and allow theoperator to select a desired mode.

In some embodiments, the system 100, e.g., the various control units 25,may communicate with, or be integrated with, a “concierge” typetelematics system, such as is operated by OnStar or ATX. Therefore, itis also possible to fuse the data from vehicles involved in an accidentor incident with that from a fixed infrastructure. Likewise, it ispossible to use vehicular sensors as a part of the monitoring system, inwhich case the GPS location data becomes a critical part of the datarecord. Currently, some vehicle navigation systems trigger an emergencycall when the airbags are deployed. As in-car telematics systems evolve,the occurrence of an airbag deployment (or other indication of anaccident) on a vehicle may be used to trigger a signal to activaterecording at any control units 25 within the proximity of the signal,and this may become a feature in some of these telematic systems whichcan be employed by the present invention to further enhance thefunctionality of the system 100.

The initial data transmission can also include the telephone number ofthe emergency response authority for that particular scene. In thisevent, the number is stored in memory within control unit 25 andcorresponds to the emergency dispatch unit closest to scene 1 asdirected by local authorities. The number of the emergency responseagency can also be stored at the monitoring center and displayed at theremote monitoring center 45 based on the location of the accident.

After the operator at the monitoring center 45 has contacted theappropriate authorities and dispatched the appropriate response, theoperator can instruct the system to initiate an upload of the at least aportion of the stored accident-related data onto a server or other datastorage device for archiving, and for later distribution to interestedparties such as the authorities, accident victims and their insurancecompanies. This uploading process can also be automated so no operatorintervention is required, and can also be dynamic so that it takes placewhile the operator is contacting the emergency response agency. The datacan be archived in a sufficiently reliable form for use in court orother proceeding as necessary. For example, the data may be watermarkedand/or associated with a hash, or a digital signature to assure that thedata is not altered and is complete. With reliable capture andsubsequent availability of audio and video evidence provided by thepresent invention, contests over liability from traffic accidents andthe associated burden on the legal system and insurance companies may besubstantially reduced.

In the preferred embodiment, video and audio compression techniques aregenerally used to compress the recorded data in order to transmitgreater amounts of information in less time using less bandwidth. Forexample, the data may be transmitted using one of the ITU multimediacommunication standards, such as h.324M, h.263, or the like. Othersuitable formats include MPEG4, AVI, WMV, ASX, DIVX, MOV(QT), etc.However, uncompressed data may also be transmitted.

In motion vector-based video compression formats, the motion vectors mayadvantageously also be used for video analysis. In particular, onecharacteristic of an accident is that one vehicle transfers its momentumto another. Therefore, by analyzing motion vectors for rapidacceleration of objects, i.e., >2 g, one may infer that thisacceleration is due to an impact, since the normal adhesion limits oftires are limited to <1.5 g. Advantageously, the motion vectors arecomputed once for both video analysis and video compression.

Once it is confirmed, either by the operator at the monitoring center 45or by automated process, that the accident-related data has beensuccessfully transmitted and uploaded, a signal is sent to the controlunit 25 to clear the memory and storage and the control unit 25 returnsto its standby state to continue monitoring the desired location 1 foranother indication of an accident. This signal can be sent automaticallywhen the system determines the transmission and receipt of theaccident-related data was successful, can require the operator toconfirm successful transmission and receipt, and to initiate sending ofthe signal manually, or can take place within the control unit 25 whenthe control unit 25 determines the transmission and receipt of theaccident-related data was successful. Either way, the system 100 isprogrammed so the accident-related data cannot be deleted until it issuccessfully transmitted to, and receipt of the data confirmed by, thedata storage facility at the monitoring center 45 or other location.Once this signal is sent and received by the control unit 25, thecontrol unit 25 resumes monitoring the desired location 1 to wait forthe next qualifying sound (or preliminary and qualifying sounds inalternate embodiments).

In one embodiment, during the transmission and/or uploading of data, thecontrol unit 25 is capable of detecting further accidents. Microphones15 are constantly monitoring sounds and comparing the signals topatterns of particular events of interest, or simply compared againststored threshold acoustic levels, to determine if preliminary orqualifying sounds are detected. Should the control unit 25 detectanother preliminary or qualifying sound during data transmission, thenew accident related data is stored in a separate file for as long asthere is storage capacity to do so, and the monitoring center 45 isnotified of the new accident over the communication link 50. Therefore,in this embodiment, a control unit 25 is capable of detecting andrecording accident-related data from multiple accidents even during thetransmission of prior accident-related data. When the stored data fromthe first accident has been successfully transmitted and received, thedata from the subsequent accidents is then transmitted to the monitoringcenter 45 in the same manner as was the first accident related data.

The present invention is not limited to any particular algorithm for theanalysis of audio and/or video data, and indeed the processor may be ofa general purpose type, which can employ a number of differentalgorithms and/or receive updates through the communication link tomodify, adapt, update, or replace the algorithm(s). Without limiting thescope of the invention, Baysian probabilistic processing, Hidden MarkovModels, and wavelet-based processing are preferred methods for acousticanalysis to determine a likelihood of occurrence of an event, such as anaccident.

It is also noted that there are types of traffic incidents which do notcomprise accidents, and indeed may have causation without respectivefault or liability. In such instances, the processor may be used todetect and classify these various incident types and report them to thecentral monitoring center 45. In these instances, the retention of arecord of the conditions may be controlled manually by an agent at thecentral monitoring center 45, or according to an algorithm specific forthese types of incidents.

According to another embodiment of the invention, a large volume of rawsensor data is accumulated, either at the location (i.e., the localcontroller) or the remote location (i.e., the central monitoring center45), for use in adapting algorithms to achieve optimum detectioncharacteristics. Therefore, according to this embodiment, while therecords need not be stored in a manner required for forensicauthentication to be used as evidence in a legal proceeding, that is,with high reliability so as to ensure that the record has not beentampered with or altered, there are stored regardless of whether theyappear to represent an incident or not (although a thresholding functionmay be applied to limit the storage or data storage requirement ofsignals which appear to represent unremarkable events).

In an alternate embodiment, the control unit 25 continues recording atleast video images after the first accident until the scene is cleared,and any subsequent accident will be captured in the running video. Inthis embodiment, the operator at the monitoring station 45 can be givena visual and/or audio cue advising that another accident has occurred,and the time of detection can be recorded for each qualifying sound andif applicable, preliminary sound, thereby giving a time record of anysubsequent accident. Alternatively, the time can be recordedcontinuously, or at specified intervals in running video.

During normal operation, the control unit 25 and other relatedcomponents are powered via the current-carrying conductors available atmost intersections and roadway locations. In an alternate embodiment, abattery backup system takes over during power failures and allows thecontrol unit 25 and other components to operate until electricaldistribution to the scene has been restored. In other alternateembodiments, the control unit 25 or components may be powered solely bybatteries which are kept charged by solar panels or other means forcharging batteries when no electricity is available, for example a windpowered generator. When under battery power or otherwise powerconstrained, the control unit 25 preferably adopts a power efficientoperating mode, for example, minimizing power hungry data capture anddata transmission unless triggered by a qualifying or preliminary(preceding) sound pattern. This power efficient operating mode cancontinue to be used while recording and transmitting accident-relateddata by minimizing the amount of video captured. One method foraccomplishing this is to reduce the resolution of the video beingrecorded and/or the number of recorded frames either consistently, or ata variable rate. When using a variable rate while waiting for aqualifying sound, the system can record at a reduced frame rate,increase the frame rate temporarily upon detection of a qualifyingsound, and return to the reduced frame rate after a predetermined lengthof time, such predetermined length of time to be determined according tothe preferences of the agency deploying the system. The connection overthe communication link 50 can also be closed as soon as the initialaccident data is transmitted to the monitoring station 45, and thenreopened later to transmit the accident-related data. Finally, therecording can be stopped at a predetermined time after a qualifyingsound has occurred instead of continuing until being reset as in thepreferred embodiment. These methods create a record of theaccident-related data that is still reliable, but occupies less storagespace and takes less transmission time, resulting in less powerconsumption.

In the preferred embodiment, the control unit 25 can be programmedremotely from the monitoring center 45 to input identification data,program contact information for the monitoring center 45, adjustrecording times and other parameters that are critical to the operationof the control unit 25 and its components, and to perform diagnostics todetect failures and to reset the control unit 25 if necessary. In someembodiments, the operator at the monitoring center 25 can send a commandto initiate recording, terminate a recording prior to the predeterminedtime, or to extend the recording to continue beyond the predeterminedtime.

In an alternate embodiment, the status of each traffic light 30 (red,green, yellow) is determined by electrically connecting the controlmeans for the traffic signal 36 to the control unit 25 so that when apreliminary or qualifying sound is detected, the control unit can recordthe state and time of change of each traffic signal 30 at the relevanttimes, and if desired the time and state of each transition of thetraffic signals' status for a specified period of time after detectingthe qualifying sound. This data may become part of the accident-relateddata that is stored and subsequently transmitted to the monitoringstation 45.

In variations of the preferred and alternate embodiments, a visualsignal can be placed at the desired location to indicate that anincident has been detected at that location, and if desired, whencontact is established with the remote monitoring center another signalcan be employed to indicate the incident has been reported. The firstsignal alerts drivers in the area that an incident has been detected atthe location, and the second that the incident has been reported,thereby reducing the demand on the resources of the emergency responsesystem that comes from multiple reports of the same incident.

Referring now to FIG. 3, a flowchart is shown illustrating the stepsperformed by the preferred embodiment of the present invention. In step51 the control unit 25 is activated and microphones 15 are sending audiosignals of sounds from the desired location 1 which are being receivedby the control unit 25, which is also receiving video signals of imagesfrom the at least one camera 35 at the desired location 1 and time andposition information from the GPS receiver that is receiving signalsfrom one or more GPS satellites 40.

While storing at least video data, (and other accident related data suchas audio, time, location and traffic signal status, as may be desired bythe agency deploying the system 1), in a circular buffer that goes onfor a predetermined period of time step 52, (said predetermined periodof time, and others referenced herein, being set in accordance with thepreferences of the agency deploying the system), the processor in thecontrol unit 25 compares incoming sounds to a database of thresholdacoustic signatures step 53 to determine if a qualifying sound ispresent in the incoming audio stream indicating a probability that anaccident is occurring. In a preferred embodiment, the control unit 25predicts traffic incidents based on both a predetermined set of acousticcriteria, as well as adaptive and possibly less stringent criteria. Thecontrol unit 25 may receive updates to its database and algorithmsthrough the one or more available communication link(s) 50.

If at any time, the incoming audio signals are determined to be aqualifying sound, the control unit 25 stops overwriting and preservesthe data stored in the circular buffer prior to the qualifying sound 54,and moves to step 55 where the control unit 25 continues to save atleast the subsequent video data, and if desired some or all of otheraccident-related data such as audio data, traffic signal status, timeand location data, (collectively referred to as the “accident-relateddata”), all of which continues to be saved in the buffer for apredetermined period of time, that predetermined period of time beingset according to the preferences of the agency deploying the system.

Also upon determination of a qualifying sound, the control unit 25starts a process to initiate contact with the monitoring center 45through the communication link 50, step 75. If contact is notestablished with the monitoring center 45, on the first try, the controlunit 25 continues to maintain the stored data in the buffer andcontinues to attempt establishing contact until contact is establishedstep 76.

Upon establishing contact with the monitoring center 45, step 76, thecontrol unit 25 transmits at least the location data, and if desired, atleast one image of the desired location 1 to the monitoring center 45step 77, which are preferably displayed on a monitor for a live operatorat the monitoring center 45 or other remote location. During thisprocess, the control unit 25 continues saving the desired accidentrelated data 78 until one of the predetermined time has passed, memorycapacity has been reached or a signal is received to terminate thesaving step 79.

When one of the predetermined time has passed, memory capacity has beenreached, or a signal received to terminate the saving step 79, theaccident-related data that has been stored in the buffer in the controlunit 25 can be transmitted at step 81, via wireless or hard-wiredcommunication link 50, to a location such as the monitoring center 45 orother remote location to be saved as a permanent record. Thistransmission can be started automatically, or by command from themonitoring center 25, and can commence after recording has finished, asin the preferred embodiment step 81, or alternately starts while thesystem is still saving accident-related data in step 78. Transmission ofthe accident related data step 81 continues until the control unit 25receives verification that the accident-related data has beensuccessfully transmitted, step 82. If the transmission step 82 is notsuccessful on the first or subsequent tries, the control unit 25continues transmitting 81 the accident related data until successfultransmission is verified 82.

The use of the term “transmission” is not meant to imply that thecontrol unit 25 must physically transmit the accident-related data, butrather indicates that the accident-related data is being passed from thecontrol unit 25 to the monitoring center 45 or other remote locationover the communication link 50 by whatever means are available forcopying or moving data from one location to another. In the preferredembodiment, the accident-related data can either be transmitted from thecontrol unit 25, or uploaded from the monitoring center 45 or otherremote location, so long as the end result of the data being stored in apermanent record at a remote location is achieved. Likewise, theverification of successful transmission can be done by the control unit25, or can take place at the monitoring center 45 or other remotelocation, and in case of the latter a confirmation signal is sent to thecontrol unit 25 indicating successful transmission.

When the control unit 25 receives verification 82 that theaccident-related data has been successfully transmitted, thetransmission is ended step 85, the buffer or memory and storage in thecontrol unit 25 is flushed 90 and processing returns to step 51 to waitfor detection of another qualifying sound. If desired, the control unit25 is reinitialized at step 99, however, this reinitialization 99 may beoptional, since in some embodiments, the control unit 25 may supportmultitasking and automated task initiation and termination.

The following describes an alternate embodiment in which recording ofaudio and video data starts only upon the detection of preliminarysounds or of qualifying sounds if no preliminary sounds are detectedpreviously. Referring now to FIG. 4, a flowchart is shown illustratingthe steps performed by an alternate embodiment of the present invention.The system is activated and the control unit 25 receives audio signalsfrom at least one microphone 15, video signals from at least one camera35, and time and position information from a GPS receiver which isreceiving signals from at least one GPS satellite 40, step 50.

The control unit 25 compares incoming sounds to a database of exemplaracoustic signatures and performs algorithms to detect trafficincident-related acoustic emissions to determine the presence of eitherpreliminary sounds, (for example, sounds of tires screeching orskidding), indicating that an accident is about to take place, orqualifying sounds (for example, sounds of two automobiles colliding)indicating an accident is occurring, step 51. Thus, in a preferredembodiment, the control unit 25 predicts traffic incidents based on botha predetermined set of acoustic criteria, as well as adaptive andpossibly less stringent criteria. The control unit 25 may receiveupdates to its database and algorithms through the one or more availablecommunication link(s) 50.

If at any time, any of the incoming audio signals are determined to be apreliminary sound 54 or qualifying sound 55, the control unit 25 startssaving in a buffer at least video signals, and if desired any one ormore of audio signals, time and location data, and data on the state ofthe traffic signals, collectively referred to as the accident-relateddata. This saving of accident related data commences at step 60 iftriggered by preliminary sounds step 54, or commences at step 70 iftriggered by qualifying sounds step 55. If the sound that triggers theprocess of saving is a preliminary sound 54, the control unit 25continues this process of saving while continuing to analyze incomingaudio signals for a match to a qualifying sound 61. This process ofsaving continues until the earlier of the detection of a qualifyingsound, or the passing of a first predetermined period of time withoutdetection of a qualifying sound, step 62. This first predeterminedperiod of time and other predetermined periods of time are set accordingto the preferences of the agency deploying the system.

On the other hand, these time periods may be adaptively determined, orcontext dependent. That is, the amount of time the system waits may bedependent on the type of preliminary sound detected, its intensity, orother sounds detected in temporal proximity. The system may also beweather and/or time-of-day dependent, since traffic incidents may bemore likely under some circumstances than others. By carefully tuningthese parameters, the sensitivity and selectivity of the system may bemaintained at a high level. Since the acoustics and dynamics of eachtraffic intersection may differ, the criteria applied by each controlunit 25 may also differ.

When the process of saving was triggered by a preliminary sound, if thefirst predetermined time passes without detection of a qualifying soundin step 62, this indicates that an accident has probably been avoided.If desired, at this stage in step 69, the data recorded following apreliminary sound can be transmitted to a remote location for lateranalysis. Otherwise, the buffer is flushed in step 90 and the systemreturns to step 50 to wait another preliminary or qualifying sound. Ifdesired, the control unit 25 is reinitialized at step 99, however, thisreinitialization 99 may be optional, since in some embodiments, thecontrol unit 25 may support multitasking and automated task initiationand termination.

Whenever a qualifying sound is detected without a preceding preliminarysound, step 55, the process of saving commences immediately upondetermination of the qualifying sound, step 70. When the process ofsaving is triggered by a preliminary sound and a qualifying sound isdetected within the first predetermined period of time, the process ofsaving continues, step 70. After determining a qualifying sound, andcommencing or continuing the process of saving 70, the process moves tostep 75 where the control unit 25 initiates contact with the monitoringcenter 45 through the communication link 50.

If contact is not established with the monitoring center 45, the controlunit 25 continues to attempt contact until contact is established, whileretaining the data saved in the buffer.

Upon establishing contact with the monitoring center 45 at step 76, thecontrol unit 25 transmits at least the location data, and if desired atleast one image of the scene to the monitoring center, step 77, whichare preferably displayed on a monitor for a live operator.

During the process of establishing contact with the monitoring center45, the control unit 25 continues the process of saving theaccident-related data, step 78 until the second predetermined period oftime has passed, storage capacity is reached, or a signal is received toterminate the process saving, step 79.

When one of the conditions in step 79 is met, the process of savingstops, step 80, and at least a portion of the accident-related data thathas been stored in the buffer in the control unit 25 is transmitted oruploaded at step 81, via wireless or hard-wired communication link 50 toa predetermined location, which can be the monitoring center 45 oranother remote location, to be saved as a permanent record. This processcan be started automatically, or by command from the monitoring center45, and can commence after the process of saving has finished, or startwhile the system is still in the process of saving accident-relateddata. The process of transmitting or uploading 81 continues untilverification of successful transmission or upload, step 82.

Upon verification of successful transmission or upload 82, the buffer inthe control unit 25 is flushed, step 90 and the process returns to step50 to wait for detection of another preliminary or qualifying sound. Ifdesired, the control unit 25 is reinitialized at step 99, however, thisreinitialization 99 may be optional, since in some embodiments, thecontrol unit 25 may support multitasking and automated task initiationand termination.

FIGS. 5 and 6 show a block diagram and flow chart or operation of asystem according to the present invention. As shown in FIG. 5, amonitoring system 200, receives input from one or more acoustic inputs201, 211, which are, for example, microphones, and one or more imagingdevices 202, 212, which are, for example, photographic cameras, digitalcameras, or video cameras. The microphones and cameras are disposed toreceive signals from a location 230, which is a scene of a potentialtraffic accident or other incident. The monitoring system 200 isinterfaced with a traffic signal control device 207, to transmit inputsthereto and/or receive outputs therefrom. The monitoring system 200generally receives power from a fixed infrastructure connection, but mayalso include a battery backup 210. The monitoring system 200 has ageolocation system or other means by which data representing thelocation can be determined or maintained, for example by satellitegeolocation (e.g., GPS), network location, or other method such as alocation code, number or equipment identifier. Typically, a GPS systemand receiver 208 are used, as this is cost efficient, requires nospecial programming, and is less prone to human error. At least videodata, and if desired other data including audio, location, time andstate of traffic signal(s), are generally stored in a memory, which hasa portion organized as a circular buffer 203, which allows asynchronousreads and writes, while maintaining a generally fixed period of storage.In a circular buffer 203 configuration, new data overwrites older dataafter a fixed period. Where reason exists to preserve the contents ofthe circular buffer 203, for example when an accident or incident isdetected, or data reliably associated with a prospective accident orincident is detected, the data in the buffer may be transferred to othermemory, or the buffer organization altered to prevent overwriting. Themonitoring system 200 may also include an enunciator, such as a light218, to indicate to persons at the location 230 that an accident orincident has been detected and/or reported to a remote locationmonitoring center 205. This enunciator or light 218 may have twodifferent states, one indicating an accident or incident has beendetected, and another indicating it has been reported. If the enunciatoris a light 218, a second light 219 may be added, one being used toindicate detection, the other to indicate reporting. When a light(s) 218(and optionally 219) is used for an enunciator, it is ideally visiblefrom a distance, acting as a signal to approaching traffic to provide awarning indicating the presence of an accident or incident at thelocation 230. The monitoring system 200 may include a transceiver 231,e.g., a radar or LIDAR transceiver, adapted to capture incident-relatedsignals at the location 230.

The monitoring system 200 communicates with the monitoring center 205through a primary communications link 204, and may also communicatethrough a secondary communications link 209. Either of thecommunications links 204, 209 may be linked to the Internet 229,although any such communications are preferably secure. The monitoringcenter 205 may communicate with other monitoring systems 226 throughcommunications links 214, 224, and the monitoring system 200 maycommunicate with alternate monitoring centers 225. Each monitoringcenter 205, 225 may have one or more live operators 217, 227, whichinteract through terminals 216, which, for example, display maps showingthe location of a monitoring system 200 producing an output, and ifavailable at least one image from the location 230. The live agents 217,227 can communicate with each other, emergency services, and locationresponders through communications systems such as telephones 215, or thecommunications can be directly integrated into the communications links204, 209, especially through the Internet 229.

As shown in FIG. 6, the method according to the present inventionpotentially includes a number of optional and alternate steps. In orderto detect an accident or incident, acoustic waves having a signaturepattern corresponding to an incident type are detected 301. Conditionsat the location are analyzed 302, which may include audio and/or videodata, other sensor data, and may encompass high level analysis. A likelyoccurrence or imminent occurrence of a vehicular accident or otherincident is detected 303. Optionally, a compliance with traffic controlregulations of vehicles at the location is determined, for example byvideo analysis of vehicle movements over time 304 or the passing of avehicle through an intersection from a direction contrary to the currentstate of the traffic signal at an intersection, and the videoidentification of a vehicle and or driver. At this stage, potentiallybefore an accident or incident has been detected or has actuallyoccurred, at least one image (from one or more cameras, simultaneouslyor polled) and other sensor data, such as sounds, traffic signal controldevice status, GPS location and timecode, are captured 305, and thenstored 306. The location and at least one image may be initiallycommunicated to a remote monitoring center, for example to assist indetermining the nature and severity of the accident or incident 307.After capture of the initial image 305, a stream of images, along withaudio, timecode, state of traffic signal, GPS (location) codeinformation continue to be captured 308, until a cessation condition ismet. Sensor data may be optionally used to model the location 309, inorder to more efficiently communicate it and/or to assist in analysis.Communications with a traffic signal control device 310 may be used todetermine its status, to implement a mode suited to the existence of atraffic incident, or to program the traffic signal control device. Acommunication pathway is established (if not preexisting), and thestored initial images, captured stream of images and otherincident-related information 306 and 308 are communicated to a remotelocation 311. The communication process continues until verification ofsuccessful communication 312, otherwise the communication is retriedand/or a new communications pathway is established 313. The storedimages and information from 306 and 308 are preserved 314 until at leastverification of successful communication. At the remote monitoringcenter, information is received and displayed on a map display,typically from a plurality of locations 315. In displaying theinformation, it may be routed to an available live agent, in a processwhich coordinates multiple communications 316. Information that has beencommunicated from the location in 311 is preferably preserved in aforensically reliable record, that is, it has sufficient reliability tobe used as evidence in a court of law 317, although if desired therecord may be preserved without forensic reliability. A forensicallyreliable record is not only reliable with respect to accuratelyrepresenting the conditions at the location, but also preferablyprovides a chain of custody to ensure that it is not altered aftercreation. The remote monitoring center may communicate with thelocation, to provide audio communications, control and program thetraffic signal control device, control and program components of thesystem, and to activate a visual alert, e.g. to indicate that anincident has been detected 318.

FIG. 7 shows a schematic view of an MITS (“MITS”) Module 110, showing asingle board computer main module 101 and open slots for optionalmodules 102, 103, 104, 105, 106, 107, 108. While the optional modulespreferably interface with the main module 101 by means of a system bus111, such as PC/104, other types of communications may be used, such asRS-232, RS-422, RS-485, CANbus, I²C bus, PXI bus, USB 2.0, IEEE-1394(Firewire), Ethernet (any type), SCSI-II, IEEE-488 (GPIB), etc.

The components of the MITS are contained in an environmentally sealedenclosure, or the MITS may be placed inside a building adjacent to thedesired location, or within another device such as a traffic signalcontrol device that is already environmentally protected. If the MITS isoutdoors and has external devices connected through one or more ports,such connections are also environmentally resistant.

As shown in FIG. 10, the MITS 110 has a GPS receiver 238, communicatingwith at least one satellite 240, and generally with a constellation ofmultiple satellites 240, 241. Further, the satellites 240, 241 need notbe part of the GPS system and can be part of any satellite navigationsystem, and communications with the satellite(s) 240, 241 may bebidirectional, and may comprise one of the possible communication links.In place of GPS, the MITS 110 may also utilize a terrestrial geolocationsystem, or use pre programmed location identifier. The MITS 110 istypically installed in a fixed position, and communicates withstationary communications infrastructure, for example, one or more of achannel switched communication system, such as the “public switchedtelephone network” (“PSTN”), a dedicated telephone connection, or othercommunication means such as T-1 or fractional T-1, frame relay, DSL,Ethernet, fiber optic cable, cellular link or cellular data link, ad hocnetwork, mobile ad hoc network, radio or microwave link, WiFi, WiMax, orother types of communications. Preferably, at least two externalcommunications options are available, which may be the same ordifferent, to provide a backup link in case of failure or unavailabilityof the primary link. In this case, it is preferred that the primary andsecondary link have different susceptibility to various impairments. Thesecondary link may be, for example, a more expensive communicationsmodality, such as cellular or satellite phone system, rather than theprimary link, which may be relatively inexpensive, but subject tovarious types of communications impairments, such as a cable modem.

The MITS 110 is connected to an audio input 234 and a video input 235.For example, the audio and video may be acquired by separate cameras 235and microphones 234, as depicted, although advantageously, a camera andmicrophone may be provided in a single module, such as anenvironmentally protected camera-cell phone which may optionally includeWiFi and GPS 236. As discussed with respect to FIGS. 11A, 11B and 11Cbelow, this arrangement allows use of relatively inexpensive hardwarewhich can be readily replaced, repaired or exchanged. In the embodimentshown in FIG. 11C, the processor of a cell phone 236′ having GPS,cellular transceiver, WiFi and/or WiMax, camera, microphone, expansionslot, and a data port (e.g., USB 2.0 or Firewire) is programmed toexecute all of the functionality of the MITS of various otherembodiments hereof, and therefore dispenses with the requirement foradditional hardware. This cell phone 236′, for example, can beprogrammed using the Java language or other suitable programming andexecution environment.

The MITS 110 preferably includes, separate from remotetelecommunications facilities, a wireless local area network accesspoint 237, also known as a hotspot, and may include a low power cellularbase station transceiver, e.g., 3G 236. This access point 237 may beIEEE 802.11 a/b/g or R/A (DSRC) compliant, or an 802.16x (WiMax) radio,although the particular standard is less important thanstandards-compliance and market acceptance. The purpose of this wirelesslocal area network access point 237 is to provide third party access tothe remote communications link or local area network 209, for exampleInternet access, although there may also be integration with thefunctionality of the MITS 110 to provide assistance with vehiclenavigation, routing or entertainment, accident detection and/orprevention, intelligent traffic control device operation, emergencyvehicle priority, traffic information systems or other function. Thelocal area network 209 may also be used as the primary or backupcommunication link for the MITS 110.

It is expressly noted that in one embodiment according to the presentinvention, at least one optional module is provided in the MITS 110which is not necessarily predefined during design or manufacture, andthe wireless local area network 209 may functionally integrate with thisoptional module as well.

While the wireless local area network access point 237 will generallyservice vehicles, there is no particular limitation on servicingstationary devices as well, such as nearby residences and businesses,which can generate revenues for the agency deploying the system whichcan come from user fees to help defray the cost of the MITS 110. On theother hand, especially for the primary communications link, the MITS 110may make use of third party telecommunications. That is, wireless localarea network 209 access provided by one or more neighbors may be used tocommunicate through the Internet to the remote monitoring center 205,with the secondary communications link employed if this primary link isunavailable or if it is otherwise necessary to maintain thefunctionality of the MITS 110 when system resources are overburdened.

To the extent that Internet or other telecommunications access (such astelephony or voice-over internet protocol (VoIP) is provided separatelyfrom the core functionality of the MITS 110, and indeed even for corefunctionality, the MITS 110 may account for usage and charge useraccounts, to provide access control, cost recovery, and potentiallyprofit. This cost recovery or profit motivation may accelerate adoptionby municipalities and other agencies, which will then promote usage, andultimately, value, since units of a widely deployed system have morevalue than sparsely deployed units. The usage accounting may beperformed by a sponsoring municipality or agency, or by an independentservice. In some instances, it may even be desirable to provide Internetaccess at no cost, as a public service to residents, and as an incentivefor business to locate in the desired coverage area.

A widely dispersed network of access points 237 may implement a mobiletelecommunications protocol, such as IETF RFC 3344 (Mobile IP, IPv4), orvarious mobile ad hoc network (MANET) protocols, 2.5G or 3G cellular, orother types of protocols. Preferably, the protocol allows the client tomaintain a remote connection while traversing between various accesspoints 237. See, U.S. Pub. App. No. 20040073642, expressly incorporatedherein by reference. Mobile Internet Protocol (Mobile IP or MIP, in thiscase, v4) is an Internet Engineering Task Force (IETF) network layerprotocol, specified in RFC-3344. It is designed to allow seamlessconnectivity session maintenance under TCP (Transmission ControlProtocol) or other connection oriented transport protocols when a mobilenode moves from one IP subnet to another. MIPv4 uses two networkinfrastructure entities, a Home Agent (HA) and an optional Foreign Agent(FA), to deliver packets to the mobile node when it has left its homenetwork. MIPv4 also supports point-of-attachment Care-of Addresses (CoA)if a FA is unavailable. Mobile IP is increasingly being deployed for2.5/3 G (2.5 or third generation wireless) provider networks and may bedeployed in medium and large Enterprise IEEE 802.11-based LANs (LocalArea Networks) with multiple subnets. MIPv4 relies on the use ofpermanently assigned “home” IP addresses to help maintain connectivitywhen a mobile device connects to a foreign network. On the other hand,IPsec-based (Internet Protocol Security, a security protocol from IETF)VPNs (Virtual Private Networks) use a tunneling scheme in which theouter source IP address is based on a CoA at the point-of-attachment andan inner source IP address assigned for the “home” domain. In general ifeither address is changed, such as when the mobile node switches IPsubnets, then a new tunnel is negotiated with new keys and several roundtrip message exchanges. The renegotiation of the tunnel interferes withseamless mobility across wired and wireless IP networks spanningmultiple IP subnets.

The MITS 110 may also include other sensors 242, such as weather (e.g.,rain, fog, sleet, snow, temperature, humidity, etc.).

The MITS 110 preferably provides a communications link for public safetyand emergency vehicles. For example, a police officer can use a personaldigital assistant-type device 244 with wireless local area networkcapability to control a traffic control device at an intersection. Whilethis communications link may comprise the generic communications linkand/or wireless local area network access point 237 heretoforedescribed, the MITS 110 may also communicate using licensed publicsafety spectrum or otherwise integrate with communications systemsprovided in public safety or emergency vehicles, without necessarilyusing the normal wireless network. For example, if the normal wirelessnetwork typically operates near capacity, or the network operates usingunlicensed spectrum, the quality of service for emergency use may beinsufficient. Access security may be controlled by username/password,virtual private network or encrypted communications, remote or biometricauthentication, or other known security techniques.

As discussed above, the MITS 110 may be remotely controlled, and thus,for example, an external signal may be used, through the MITS 110, tocontrol the state of a traffic signal through the traffic signal controldevice, to control microphones 234, cameras 235 and data storage, andtelecommunications functions, to control an optional module using astandard application programming interface (API), or the like. In thecase of accident detection or prevention functions, it is also possiblefor the MITS 110 to coordinate functions of vehicles approaching anintersection. That is, in order to prevent an accident, or in variousother circumstances, it may be appropriate for the MITS 110 tocommunicate with nearby vehicles to control their behavior, eitherdirectly through an automated control system in the vehicle, orindirectly, by providing individual instructions to a driver. Thus, forexample, an intersection might be normally marked “No Turn On Red”, butthe MITS 110 may provide permission to a compatible vehicle tonevertheless proceed. By providing vehicle-by-vehicle permission, afiner-grained control may be executed. For example, a small car, with anarrow turning radius, might be permitted to turn right, while a truckmight not, depending on traffic circumstances. Likewise, if a vehicle isapproaching black ice near an intersection, the MITS 110 may communicatewith nearby vehicles and control the intersection to avoid the need fora change in speed by the vehicle. Using its cameras 235, the MITS 110may have a unique perspective regarding the relationship of vehicles,and may therefore execute emergency measures on their behalf to avoidcollision or other types of incident.

Intelligent transportation systems typically gather and have updated andprecise traffic information available to them, and the MITS 110 mayutilize and distribute this information. This information itself may bevaluable to third parties. For example, U.S. Pat. Nos. 6,252,544 and6,429,812, expressly incorporated herein by reference in their entirety,disclose, for example, a system in which in-vehicle navigation systemsmake use of remotely transmitted real time traffic information to guidea car to avoid traffic. Likewise, Navteq and XM Radio have developed aservice called NavTraffic for providing traffic information by satelliteradio to an in-vehicle navigation system. In each of these cases, thedata available from the MITS 110 may be provided to these systems, forrelay to the vehicles. Likewise, information from vehicles, such asvideo data, on-board diagnostic information, or various vehicle data,may be communicated to the MITS 110 for its own use as well as fordissemination for use by others.

In order to encourage usage of aspects of the system, it may bedesirable to protect the privacy of vehicles and their drivers, and anumber of means are available for this purpose. For example, unlesscertain conditions are met, the MITS 110 may censor communications toblock transmission of license plate data, while this data may beretained internally or encrypted for transmission externally. Likewise,certain sensitive information, such as vehicle speed (where no incidenthas occurred), may be intentionally corrupted in such a manner that itis statistically accurate but not forensically reliable, thus avoidingissues as whether interaction with the MITS 110 would result inpotential liability for traffic infractions. The basis for thecorruption may also be stored locally or encrypted and communicatedexternally, in case it is later determined that there has been anincident, or for other legitimate reasons.

The MITS 110 may also include radio frequency identification (RF-ID)technologies 243 for identifying people, vehicles or other objects. TheMITS 110 may also be used to assist in parking management. For example,the location of vacant parking spots (detected by special sensors or byvideo cameras) may be broadcast to vehicles looking for parking. Parkingregulations may also be enforced by similar means. The MITS 110 may alsobe used to account for parking usage, alleviating the need for metersand coin or bill feeds, for example expanding electronic toll collectionor similar technologies for payment of parking fees.

The MITS 110 can further assist in public safety, for example warningmotorists of potential hazards, such as children playing nearby orbicyclists, and possibly regulating traffic speed based on localconditions. By monitoring activities with video, various criminalactivities can be recorded, and the criminal(s) identified. For example,the system may be used to capture evidence of abduction or othercriminal activities.

As shown in FIG. 8, the optional module 112 need not be a bus-baseddevice, and may be located within the enclosure by as a separatesubsystem. Likewise, as shown in FIG. 9, the optional module 114 maycommunicate with the main module 101 by means of a data communicationsconnection 113, which may be, for example, a thin Ethernet connection,Firewire, USB, serial interface such as RS-422, parallel interface, suchas IEEE-488, wireless communications such as IEEE-802.11g, Bluetooth,DSRC, and fiber optic cable. The MITS 110 may support multiple modulesof this type, and may utilize the data communications connection 113 toconnect the microphones, cameras, access points and other externaldevices.

FIG. 10 shows various connections and connectivity options for the MITS110 according to the present invention. The MITS 110 preferably includesan IEEE-802.11x, preferably multistandard (a/b/g/R/A (DSRC)), hotspotaccess point 237, which supports MobileIP and can communicate withwireless enabled devices 244 such as a pocket PC, tablet PC, networkenabled cellular phones, or notebook computers equipped with wirelessEthernet. This access point 237 may also support MANET networkarchitectures, and provide both public and private access. The MITS 110may be connected to an Ethernet network 209 T1, DSL or Cable Modem forbroadband connectivity. The MITS 110 is preferably linked by any ofvarious means to a central monitoring station 205, which typicallycomprises a call center, having agent terminals 247 and agent voicecommunication telephones 248, as part of a vehicular incident managementor telematics system which also incorporates remote data storage 245 forstoring information communicated from the MITS 110. The MITS 110preferably includes a GPS receiver 238, which receives geolocationsignals from a set of satellites 240, 241, although other types ofgeolocation systems, and in particular terrestrial geolocation systemsincluding the E911 system, or a location identifier data may also beemployed. The MITS 110 may be linked to a cellular 236 or other type ofradio transceiver 237, which permits wireless communications with acellular system 246 or other wireless network 249. The MITS 110preferably has a camera 235 for receiving image data, for exampleimaging vehicular incidents or other objects. The camera 235 may have amicrophone or separate microphones may be provided for acoustic signalinput 234. The MITS 110 may also wirelessly communicate withWiFi/camera/cellular phones 236 (or the components of cellular phonesincorporated within the MITS 110), either wirelessly through thecellular 246 or other wireless system 249 antennas, the hotspot accesspoint 237, or by a wired data connection through the phones data port.The MITS 110 may utilize cellular phones 236 that are WiFi and/or GPSenabled. The MITS 110 may also be connected to or incorporate othertypes of sensors 242, and receivers for RFID transponders 243.

FIGS. 11A and 11B show aspects of another alternate embodiment of theMITS 110 intended to use off-the-shelf components for accident detectionintended to be a lower cost alternative to the preferred embodiment. Onealternate solves this problem and minimizes the cost of deployment andoperation of the system by utilizing a cellular phone and the wirelessE-911 system.

As shown in FIG. 11A, this alternate embodiment also uses at least onemicrophone 234 connected to the MITS 110, at least one of a still orvideo camera 235, and a cellular transceiver 250. Preferably, themicrophone 234, camera 235 and cellular transceiver 250 can be combinedinto one unit, as shown in FIG. 11B, as in the video-enabled cellularphones that are becoming widely available in the consumer market.Preferably, this video enabled cellular phone 236 also incorporates WiFi237 and a GPS receiver 238. For this application, preferably, the videocellular phone unit 236 is weather proof itself, or is enclosed in aweather proof enclosure that enables it to detect sounds and capturevideo of the desired location, and it is connected to the MITS 110either wirelessly or by a wired data connection through the data port onthe cellular phone 236. While it is preferable to have all thecomponents together in one unit as shown in FIG. 11B, a variation ofthis alternate embodiment shown in FIG. 11A can use the components of avideo cellular phone 236 or similar components, placing the components,234, 235, and 250, in various places about the desired location andconnecting them together either through the MITS 110 or through thecellular transceiver 250 or WiFi 237 component. The cellular telephone236 (or the components of a cellular telephone 234, 235 and 250 as thecase may be) is preprogrammed so that upon receiving a signal from theMITS 110, it places a “911” call and delivers a pre-recorded messageindicating a vehicular incident may have occurred at the location. Insome embodiments, this programming may be in the MITS 110, and the MITS110 may execute the 911 call through the cellular transceiver 250 anddeliver the pre-recorded message once connected. The cellulartransceiver 250 may also transmit an SMS or other data message in theevent that the emergency response center is data-call enabled.

In this alternate embodiment, the MITS 110 monitors the locationcomparing incoming audio signals for a match to at least one ofpreliminary and qualifying sounds as in the preferred and otheralternate embodiments of the accident detection system. When theprocessor determines that an accident is about to or has occurred(detection of a sound matching at least one of a preliminary orqualifying sound), a signal is sent to the cellular telephone 236 todial 911 (or the MITS 110 unit connected to the cellular phone 236 dials911).

The E-911 operator will receive the pre-recorded message indicating anaccident has happened at this location, and the E-911 system willprovide the location. Preferably, at least one video image is availableto the E-911 operator, although this is not required in this alternateembodiment. As in other embodiments, the images and otheraccident-related data can be temporarily stored in memory in the MITS110, or can be sent directly by the cellular telephone 236 to thedesired remote location for permanent storage.

In one variation of this alternate embodiment, if the images are storedin a memory buffer in the MITS 110, then images for a predeterminedperiod preceding and following the detection of an accident, togetherwith other desired incident information, can be automaticallytransmitted to a remote location where a permanent record of the eventis created.

This alternate embodiment uses standardized components which can bereadily replace repaired or exchanged, and employs the existinginfrastructure and resources of the E-911 system to report and determinethe location of an accident and dispatch emergency response personnel,thereby reducing the cost of operation and administration of theaccident detection system by eliminating the need for a separate remotemonitoring center staffed with manned by live operators. The incidentrecord can be retrieved from the remote location at a later time basedon the time and location or other criteria to identify the record of anincident. It is also possible in this alternate embodiment to use videoimages to determine the state of the traffic light, thereby furtherreducing the cost of the accident detection system and its deployment.While a separate communication link is desirable, in some variations ofthis alternate embodiment, it may be desirable to use the cellular phone236 itself to transmit the accident related data to a remote location,in which case, the remote location is contacted by the cellular phone236 after completion of the 911 call, and the images and other desiredaccident related data are transmitted to the remote location aftercontact is established. A wireless modem with cellular phonecapabilities may also be employed, where the 911 call is placed as acellular call and the data functionality is used to transmit the imagesand other desired accident related data.

In another variation of this embodiment, when the MITS 110 detects avehicular incident, the cellular transceiver 250 is preprogrammed orused by the MITS 110 to call 911 and deliver a pre-recorded message,without sending or saving any additional information. This embodimentsupports deployment of a lower cost yet reliable accident detection andreporting system with a single purpose of detecting and reportingaccidents to the E-911 system. In a variation of this alternateembodiment as shown in FIG. 11C, the cellular transceiver 236′ assumesthe function of the MITS (and can be treated as a MITS), and the audio,video, processor, memory, WiFi and other desired modules areincorporated into a single housing. Another variation of this alternateembodiment can utilize a regular telephone and telephone line and thepublic switch telephone network (PSTN) to achieve the same purpose.

It is understood that the present invention is not technologicallylimited by available cellular telephone hardware (or other hardware) oremergency response center implementations, and that available resourcesmay be employed in accordance with the teachings of the presentinvention. For example, a WiFi or WiMax enabled video-conference cellphone may implement some or all of the functions described with respectto the preferred embodiment, albeit in a smaller form factor withpotentially greater environmental vulnerability.

The instant invention has been shown and described herein in what isconsidered to be the most practical and preferred embodiment andalternate embodiment. It is recognized, however, that the preferred andalternate embodiment are not intended to be limiting, and thatdepartures may be made therefrom within the scope of the invention andthat obvious modifications will occur to a person skilled in the art.

What is claimed is:
 1. A method of operating a cellphone, comprising:receiving at least first audio voice information with a microphone;transmitting at least second audio information with a speaker; remotelycommunicating the received at least first audio voice information andthe transmitted at least second audio information over a digitalcellular network through a cellular transceiver; capturing a stream ofimages with a video camera; analyzing, by the processor of thecellphone, the captured stream of images to detect at least a motion ofan object represented in the stream of images, and to control at leastthe cellular transceiver; and automatically composing and transmitting,by the processor of the cell-phone, a Short Message Service (SMS) to aremote location in response to at least the received first audio voiceinformation and the analyzed captured stream of images.
 2. The methodaccording to claim 1, further comprising providing, within a cellphonedevice: the microphone; the speaker; the cellular transceiver; the videocamera; an interface to provide at least wireless Internet connectivityfor other devices through a wireless local area network; a locationdetermining system for generating data indicative of a location of thecellphone; a universal serial bus (USB) interface; an IEEE-802.11compliant radio transceiver; and a bluetooth communication interface. 3.The method according to claim 1, further comprising analyzing motionvectors of objects in the stream of images.
 4. The method according toclaim 3, further comprising predicting a collision of objects based onthe analyzed motion vectors.
 5. The method according to claim 1, furthercomprising providing a receiver for an RF-ID transponder.
 6. The methodaccording to claim 1, further comprising providing a communicationhotspot to provide wireless Internet connectivity for other devicesthrough a wireless local area network.
 7. The method according to claim1, further comprising communicating telematics information through thecellular transceiver.
 8. The method according to claim 1, furthercomprising providing vehicle traffic information to a subscriber.
 9. Themethod according to claim 1, further comprising compressing the streamof images into an MPEG-4 format; and buffering the MPEG-4 formatcompressed stream of images in a memory and transmitting the bufferedstream using a reliable protocol which ensures receipt of the bufferedstream prior to deletion of information in the buffer.
 10. The methodaccording to claim 1, further comprising: providing a cellphone havingat least: the microphone; the speaker; the cellular transceiver; thevideo camera; an interface to provide at least wireless Internetconnectivity for other devices through a wireless local area network; alocation determining system for generating data indicative of a locationof the cellphone; a universal serial bus (USB) interface; an IEEE-802.11compliant radio transceiver; and a bluetooth communication interface;communicating a voice conversation comprising the audio informationthrough the microphone, the speaker and the cellular transceiver; andselectively producing an output which is dependent on a characterizationof the motion of the object.
 11. The method according to claim 1,further comprising communicating through an ad hoc communicationsnetwork.
 12. The method according to claim 1, further comprisinganalyzing information received through the microphone to recognize atype of sound.
 13. The method according to claim 1, further comprisingconducting a videoconference using the video camera and the microphone.14. The method according to claim 1, further comprising receivingexecutable programs from a remote system through the cellulartransceiver.
 15. The method according to claim 1, further comprisingcommunicating using a voice over internet protocol communication throughthe cellular transceiver.
 16. A method for automatically operating acellphone, comprising: providing a cellphone having at least: amicrophone; a speaker; a cellular transceiver; a video camera; aninterface to provide at least wireless Internet connectivity for otherdevices through a wireless local area network; a location determiningsystem for generating data indicative of a location of the cellphone; auniversal serial bus (USB) interface; an IEEE-802.11 compliant radiotransceiver; and a bluetooth communication interface; communicating avoice conversation through the microphone, the speaker and the cellulartransceiver; capturing a stream of images from the video camera;buffering the captured stream of images; analyzing the buffered capturedstream of images to detect and characterize at least a motion of anobject represented in the captured stream of images; and automaticallyinitiating at least one Short Message Service (SMS) through the cellulartransceiver to a remote location in response to the detected andcharacterized at least a motion and a sound received by the microphone.17. The method according to claim 16, further comprising recognizing atleast one type of sound received through the microphone.
 18. A method ofoperating a cellphone having a microphone, a speaker, a cellulartransceiver, a video camera, and an automated processor, the methodcomprising: receiving at least audio voice information with themicrophone of the cellphone; processing the at least audio voiceinformation with the automated processor to classify at least a firstportion of the at least audio voice information received through themicrophone; capturing a stream of images with the video camera of thecellphone; analyzing the captured stream of images with the processor todetect and characterize at least a motion of an object represented inthe stream of images; controlling at least the cellular transceiver tocommunicate at least one Short Message Service (SMS) through a digitalcellular network using the cellular transceiver selectively independence on the classified at least audio voice information and thedetected and characterized at least the motion of the object; remotelycommunicating at least a second portion of the audio voice informationover the digital cellular network using the cellular transceiver; andreproducing at least audio information received through the cellulartransceiver with the speaker.